Product of Experts for Visual Generation
Authors: Yunzhi Zhang
Carson Murtuza-Lanier
Zizhang Li
Yilun Du
Jiajun Wu
Venue: ICLR 2026 Poster (2026)
Keywords: generative models
image generation
video generation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: An inference-time sampling framework for image and video generation that compose knowledge from heterogeneous perceptual models.

Abstract: Modern neural models capture rich priors and have complementary knowledge over shared data domains, e.g., images and videos. Integrating diverse knowledge from multiple sources—including visual generative models, visual language models, and sources with human-crafted knowledge such as graphics engines and physics simulators remains under-explored. We propose a probabilistic framework that combines information from these heterogeneous models, where expert models jointly shape a product distribution over outputs. To sample from this product distribution for controllable image/video synthesis tasks, we introduce an annealed  MCMC sampler in combination with SMC-style resampling to enable efficient inference-time model composition. Our framework empirically yields better controllability than monolithic methods and additionally provides flexible user interfaces for specifying visual generation goals.

Published as a conference paper at ICLR 2026
PRODUCT OF EXPERTS FOR VISUAL GENERATION
Yunzhi Zhang'
Carson Murtuza-Lanier
Zizhang Li" Yilun Du? Jiajun Wu
Stanford University
Haarvard University
ABSTRACT
Modern neural models capture rich priors and have complementary knowledge over shared data domains, c.g., images and videos. Integrating diverse knowl- edge from multiple sources-including visual generative models, visual language models, and sources with human-crafted knowledge such graphics engines and physics simulators remains under-explored. We propose a probabilistic frame- work that combines information from these heterogeneous models, where expert models jointly shape product distribution over outputs. To sample from this product distribution for controllable image/video synthesis tasks, we introduce an annealed MCMC sampler in combination with SMC-style resampling to enable efficient inference-time model composition. Our framework empirically yields better controllability than monolithic methods and additionally provides flexible user interfaces for specifying visual generation goals.
INTRODUCTION
Recent image and video generative models (Saharia al., 2022; Rombach al., 2022; Ho al., 2022: Brooks et al., 2024) have achieved remarkable success in realistic appearance modeling, yet still have limitations in following complex text instructions and adhering to real- well-world constraints such as physical laws. The former would benefit from semantic priors in visual language models (VLMs) (Radford et al., 2021; Li et al., 2023; Bai al., 2023; Achiam et al., 2023), and the latter from rules embedded in physics simulators. However, training single model to absorb all informa- tion sources (tcxts/visual corpora, simulation trajectories, etc.) can be prohibitively expensive.
We address this challenge by integrating knowledge across models at inference time (Du & Kael- bling, 2024), leveraging visual generative priors, discriminative rewards from VLMs, and sule-based knowledge from physics simulators. We focus the task of controllable visual generation and ag- gregate opinions across a set of "expert" models by sampling from the product distribution defined by the models. Each expert, in our case, pretrained models such VL.Ms image and video gener- ative models, focuses on one or more constraints in generation, and the resulting product distribution naturally assigns high probabilities only samples that satisfy all constraints simultaneously. The use of product distributions to combine the opinions of multiple experts has been extensively used in the past (Bacharach, 1972; Genest & Zidek, 1986; Hinton, 1999), and in the visual domain in Markov Random Forests (Wang et al., 2013; Kolmogorov & Zabih, 2002; Glocker et al., 2008; Boykov & Funka-Lea, 2006) and Conditional Random Fields (Boykov & Jolly, 2001; Kumar & Hebert, 2003; He et al., 2004).
Despite its conceptual appeal, sampling from product distributions often intractable. Straightfor- ward approaches like rejection sampling are inefficient due to vanishing acceptance rates in high dimensions. Recent works have gotten around these difficulties by either composing model distri- butions in a simpler Gaussian latent space (Huang et al., 2022) or by composing through models through an annealed Langevin procedure (Du et al., 2020; 2023; Geffner et al., 2023), where the combination of annealing and gradient- based sampling enables the modes of the product probability distribution to be effectively identified. In this paper, we provide a more general framework that en- ables to sample from the product of both autoregressive and diffusion-basee generative models, as well as discriminative models such as VLMs. Our approach relies on a combination of Annealed Importance Sampling (Neal, 2001) and Sequential Monte Carlo (Doucet et al., 2001a) to effectively find the modes of the product distribution across all experts.
Project page: https:: pettps:/prrouccee ow-eexeetaa...omo tht... 10/-

---

Published as a conference paper at ICLR 2026
Our overall framework enables to integrate multiple input constraints without costly retraining. We empirically validate its benefits in image generation following complex text instructions. also naturally yields flexible user interface for defining complex generative goals by selecting and configuring experts, e.g. allowing users to specify the pose and motion trajectory of an object of interest, insert it into existing image, and animate the full scene.
Overall, our contributions are threefold:
i) We formulate controllable visual synthesis tasks in unified Product-of -roppcttof-Experts (PoE) frame- work that enables principled knowledge integration of heterogeneous generative and discrimi- native models, well physics simulators.
ii) We propose a practical, computationally efficient sampling framework based on Annealed Im- portance Sampling (AIS) and Sequential Monte Carlo (SMC).
iii) Our method enables flexible forms of isserinstruccton-folllee wowloooigggoooiiiggggooo (texts, images, and low-level specifications such as object poses and trajectories) for image/video synthesis applications and achieves better user controllability and output fidelity compared to baseline methods.
2 RELATED WORKS
Compositional Generative Modeling. Prior works on compositional generative modeling typically combine multiple generative models to jointly generate data samples (Du et al., 2020; Garipov et al., 2023; Du et al., 2023; Huang et al., 2022; Mahajan al., 2024; Du & Kaelbling, 2024; Bradley et al., 2025; Thortoo et al., 2025; Gaudi et al., 2025) and has been previously applied in visual content generation (Liu et al., 2022a; Bar- Tal et al., 2023; Zhang et al., 2023; Yang et al., 2024; Su et al., 2024; Li et al., 2022), but such applications typically focus multiple homogeneous generator experts, or, in the case of Li et al. (2022). a single generator expert with multiple discriminative experts. In contrast, our approach provides general probabilistic framework through which many heterogeneous generative and discriminative experts can be jointly combined in generation.
Reward Steering from Diseriminative Models. Our work is further related to recent work steering generative models with discriminative models and reward functions. Methods using gradient descent on a discriminative reward (Grathwohl et al., 2019; Dhariwal & Nichol, 2021; Bansal et al., 2023; Luo et al., 2025; He et al., 2023; Ye et al., 2024; Song et al., 2023; Rout et al., 2025) assume dense, differentiable gradients on the reward. Classifier-free guidance (CFG) (Ho et al., 2020) alleviates the need for training explicit reward models (classifiers) and defer more detailed discussions to Section G. Another branch of work applies SMC for more exact reward steering (Wu et al., 2023; Zhao et al., 2024; Singhal et al., 2025). Both Skreta et al. (2025); He al. (2025) and our work build on top of this formulation and consider composing both generative and discriminative experts, but with the following difference: when considering generative expert products (Eq. (2)), these works calculate path-wise importance weights to ensure sampling correctness, risking weight degeneracy as path length grows (Skreta et al., 2025), while ours uses per-timestep MCMC kernels that leave pe (Eq. (3)) invariant without accumulating weights.
Video Generation with Physical Simulators. Our framework integrates knowledge from physical simulators to improve physical accuracy compared to end-to-end models, while alleviating the need for tedious 3D scene setups in traditional graphics rendering pipelines. Recent works have explored such direction typically convert physical simulator outputs into specific type of conditional signals such as optical flow (Burgert et al., 2025; Montanaro et al., 2024; Yang et al., 2025) and point tracking (Gu et al., 2025), or rely on a single pre-trained video generation model (Liu et al., 2024a; Chen et al., 2025; Tan et al., 2024), while our framework allows for the integration of various signals available from physical simulation, providing more complete specifications of controls.
3 METHOD
We aim to generate complex scenes x E Rd by combining the priors from both generative models (such as image/video models conditioned on texts or control signals converted from physics simu- lators) as well as discriminative models (such as VLMs), with examples detailed in Section 4. We formulate this composition of models in probabilistic manner, where generative models are repre- sented as probabilistic priors over data (p(6) (x)) Diseriminative models are written constraint
2

---

Published as a conference paper at ICLR 2026
functions, (r() R - 3- each assigning scalar reward that represents how much sam- ple x matches the constraint encoded in the model. Each constraint function is converted into an unnormalized probability distribution through the Boltzmann distribution, (z) = (expr()))xx)
To combine these two classes of models, we aim to sample from the product distribution:
M
x ~ p(z) 100 (x) (z) 9 (x)
(1)
where each generative expert (x) and discriminative expert q (x) are defined over parts of the scene x. The product distribution p(z) has high probability for sample x when has high probability under both (2) and g( (a:) (Hinton, 2002; Du et al., 2020; Du & Kaelbling, 2024), allowing us to compose together the priors in both generative and discriminative models.
However, sampling from Eq. (1) is challenging, as need to search for samples that satisfies con- straints across all experts. Below, we will discuss how we can efficiently and effectively sample from the Eq. (1) for high fidelity visual generation. We first discuss how we can sample from multiple generative experts in Section 3.1 and then discuss how to sample jointly from multiple discriminative and generative experts in Section 3.2, followed by a practical implementation in Section 3.3.
3.1 SAMPLING FROM GENERATIVE EXPERTS
We first discuss how we can effectively sample from the product of set of generative experts.
N
p(a) :x II (x).
(2
To sample from the above distribution, we can use Markov chain Monte Carlo (MCMC) Robert et al. (1999), where iteratively refine samples based on their likelihood under the product distri- bution. In discrete settings, a variant Gibbs sampling may be used, while in continuous domains, Langevin sampling based on the gradient of log density may be used (Welling & Teh, 2011).
A central issue, however, is that MCMC is a local refinement procedure and can take an expo- nentially long time for the sampling procedure mix and find a high likelihood scene from the product distribution (Robert et al., 1999). To effectively sample from product of generative experts, we propose use Annealed Importance Sampling (AIS) (Neal, 2001) and construct of a sequence of T' distributions, {pr (=)) where (r(x) is smooth, casy-to-sample distribution and p1 (r) is the desired product distribution defined in Eq. (2). To draw a sample from p1 (I), we first initialize a sample from Pr (z)) and iteratively run MCMC on the sample to sample from each intermediate dis- tribution before finally reaching Pi (z). Since intermediate distributions are easier to sample from, this enables us to more effectively find high likelihood scene from the product distribution.
We construct the intermediate probability distributions of the following form:
N II
(r( :0 P (x),
where p. (x) is an expert-specific distribution interpolating between an initial distribution (e.g.,
Gaussian or uniform) and the final expert distribution (x)).. We list concrete choices of p. (0) (x)
for two common classes of generative models in Section 3.3. For autoregressive models (Oord
et al., 2016; Larochelle & Murray, 1), it is the marginal distribution for a prefixed data region determined by t; for diffusion/flow models (Liu et al., 2022b; Lipman et al., 2022; Ho et al., 2020; Sohl-Dickstein et al., 2015), it is the time marginal distribution with estimated score functions.
Composing Conditional Generative Experts. To improve the efficacy of sampling from the prod- uet distribution of generative experts, can modify each generative expert to be conditionally dependent on the outputs of other experts. Specifically, we can represent the product distribution as
N
I ~ p(x) :
(0) (acl))(()))
(4)
3

---

Published as a conference paper at ICLR 2026
Algorithm Product Distribution Sampling
1: Input: Annealing length 7' initial distribution pr(+), particle count L, MCMC step count K, generative experts (Pr (+) (-) with kernels Keee++(- for MCMC initialization and
K,(- 1-) for MCMC steps, discriminative experts with log probabilities (-()
2: Initialize: Sample x((1 pr((), vI 1, ,
3: for = T to do
Transition to the next annealed distribution Parallel sampling MCMC initialization
4: for - 1 to L do
5:
xx(1) +- K2t (- x
6:
for 2z(1) <- K,(- (
to K do
7:
MCMC step
8: end for
9: end for
10: Resample samples with log weights Σ)=1 (10) (x(1) where r, (x = r()()))))
11: end for
12: 1* < max/ Σ j= 7()) (xx1)
13: Output: (11)
where each i; refers to the part of scene represented by the generative expert p(a, and Tpa(i) refers to the other parts of a scene specified by other experts the expert is conditioned on.
Making each generative expert conditionally dependent on the values other experts reduces the multi-modality in the distribution pt*) (ar), enabling more efficient MCMC sampling the product distribution. Its benefit is empirically shown in Fig. 2 and Table 1.
3.2 PARALLEL SAMPLING WITH DISCRIMINATIVE EXPERTS
Next, we discuss how we can modify the annealed sampling procedure in Section 3.1 sample from the full product distribution from Eq. (I), including discriminative experts. Similar to the previous section, we can define sequence of intermediate sampling distributions
N (z) P.
(x) : II
(z).
(5)
-=1
To implement AIS, one simple approach importance sampling as follows. We first obtain L sam- ples from the product of the intermediate generative expert distributions defined in Eq. (3) according to Section 3.1, and then draw a weighted random sample from these L samples, each with impor-
tance weight _ 4t (2) (1:) corresponding the likelihood under product of discriminative experts.
An issue with this simple procedure is that each sample drawn from the MCMC may be correlated with each other, since MCMC slow at mixing and may not cover the entire generative product distribution. To improve coverage, maintain particles over the course of annealing. At each intermediate distribution, we run MCMC each particle draw samples from the product of the generative distributions, and then weigh and resample particles based on their likelihood under the product of the discriminative experts.
parallel SMC sampler (Doucet et al., 2001b), where
Our overall algorithm requires only black-box access to each discriminator, requiring knowing only the likelihood the discrininator assigns to a sample æ and not additional information such as log- likelihood gradients. If we do have more information about the discrininative expert's form, can directly treat it as generative as in Section 3.1 and use approaches such as gradient- gradient-based MCMC.
3.3 FRAMEWORK INSTANTIATION
We now introduce a specific implementation of the framework as summarized Algorithm 1, which describes the annealed distribution and expert instantiations primarily used in the paper.
Generative Experts. The framework requires specifying the annealed generative distributions fac- propagate samples from the previous
tors {pt (8) (x) (Eq. (3)), kemne Ke-tt1(: | ()
4

---

Published as a conference paper at ICLR 2026
Graphics Engine Input
Graphics Engine Input
Graphics Engine Input
Pluiy CAStOry
Cryny Casgen
CASteO
Caster
Input Image
"metal box"
"glass box"
"coke can"
Figure 1: Application Image Object Insertion where the goal is to insert assets posed in graphics engine (top row) and described with text prompts (bottom row) into images (first column). nealed distribution Pc+1 (z) the next distribution pr(x)) (line 5, Algorithm 1). and a MCMC kernel K,(- 1-) that leaves Pt invariant (line 7). While Ke--== does not draw sample from pr (3) exactly, serves as an effective initialization for MCMC sampling.
When generative expert p()( is a flow model, natural choice for {pt (x) is discretization of the probability path generated by its velocity prediction v/ (x)). We set K as one Euler step in the flow ODE integration, and Kc is the Langevin dynamics transition, both under composed
(z)).
score Σ, v log _ (x) (Section C.I). When p(*) is an autoregressive model, we use its prefix
(2) (x) =0000 (1) such that æ1.J x, where F1.7+1-+ is data slice. We set K,+++1 such that appends the next data slice sampled from the model-predicted conditional distribution, and Kr can be implemented as Gibbs sampling (Section C.2).
marginal for annealing, namely Pt
Conditional Sampling. We implement conditional generative experts Pt (xt Tpa(s)) for flow models by modifying the original generative experts (1) (ar,). particular, we modify the generative flow 0f (8) (x:) to take into account the predicted flow at parent regions
vi (ara xF)) 1i
(2) (x:) W Σ
Va
( (ax) (a:) stopgrad 0z (P) (xx))
(6)
where w is the learning rate. Justifications of this approximation are deferred to Section C.3.
Discriminative Experts. We use pre-trained VLMs as reward functions (x). When generative experts flow models, the noisy samples x out of distribution for discriminative experts. We
define the intermediate counterparts in Eq. (5) as (i) (x) = (() where is the predicted endpoint of (Chung et al., 2022; Efron, 2011) whose computation is specified in Eq. (8).
4 EXPERIMENTS
We apply our framework to image/video synthesis tasks. Expert configurations can adapt to the desired, task-dependent input granularity, including high-level text instructions (Sections 4.1 to 4.3) and precise, low-level pose controls (Sections 4.1 and 4.2), and leverage expert knowledge ranging across natural image and video priors (Sections 4.1 to 4.3), precise physics rules from physical simulators (Section 4.2), and semantic visual understandings from VLMs (Section 4.3). To better quantify sampling quality, we evaluate the method on synthetic data with known ground truth target distribution in Section E.
5

---

Published as a conference paper at ICLR 2026
CocaCClla
Input Image & Rendering
Ours
RF-Solver
Ours (No Conditional)
Figure 2: Image Object Insertion Comparisons. Our method better adheres to input geometric conditions while faithfully preserving background details. The last column shows that conditional sampling improves visual harmonization and fidelity.
Controllability
Image Quality
Text Aligament
Methods
MSE (bg) (U)
LPIPS (fg)
GPT-4o (t)
Aesthetic
GPT-4o
ImageReward (f)
GPT-4o (1)
Grophicx Engine Rondering Inpur
RF-Solver
0.178=0.081 1.619:00.666
0.518+00110
0.618
0.662#0.100
0.9484+0.777
0.438=u.179
Ours No Cond
1.61110.661
0.727 0.065
0.59910.059
0.718:10110
1.11210.668
0.675=0.180
Ours
1.429 00002
0.065+0.022
0.827 +0.078
0.596+00.011
0.783 +0.082
1.175=0.000
0.817+00119
Character Rendering Iopar (Magic Insert (Ruiz et al., 2024) Datasct)
Magic Insert
0.769.10.33
0.106:0.044
0.74311.14
0.721
0.75820.130
1010
0.649=0.141
Add-it
0.710:10.475
0.128:0.008
0.763
0.7151000066
0.778:20.00
1.07810.706
0.681=0.155
FLUX-Fill
0.058
0,103
0.655+0.147
0.69140.070
0.746+0.00
+-0777
0.390=0.152
SDEdit
0.968
0.026+0.020
0.744+00111
0.724+0.074
0.871+t.00
1.640+0.376
0.658=0.144
Ours
0.365:0.247
0.084=0.033
0.81810.097
0.760 060
0.769
1.71120.308
0.763 107
Table 1: Image Object Insertion Evaluation.
4.1
IMAGE EDITING
Task. This task provides the following image object insertion interface for applications where users want to insert object precisely into certain parts of the image. Inputs consist of an image to be edited, an input 3D asset posed in a graphics engine, and a text description describing higher- level information such object materials ("metal") and semantics ("coke can"). We deploy two generative experts for this task: a depth-to-image model, FLUX.1 Depth [dev], and an inpainting model, F Fill [dev] (FLUX, 2024). The former ensures inserted objects follow input pose specifications, and the latter provides a natural image prior to produce realistic outputs.
Evaluation. We compare with an editing method, RF-Solver (Wang et al., 2024), which inverts the input image to Gaussian noise and then generates the output starting from that noise, conditioned on a text prompt that additionally describes where to insert the object, c.g., "a metal box standing on the ground". The evaluation dataset consists of 10 natural images paired with 3 object assets, resulting in 30 scenes in total, with examples in Fig. 1. The metrics cover three major aspects: con- trollability, image quality, and text alignment. Specifically, we evaluate the MSE distance between generation outputs and input images on background pixels to evaluate background preservation. and LPIPS (Zhang al., 2018) distance between outputs and graphics engine's RGB renderings on foreground pixels to measure the fidelity to input objects. We report aesthetic score using the LAsON-Acsthetic predictor (Schuhmann & Beaumont, 2022) and measure text alignment with Im- award (Xu et al., 2023), and query GPT-4o for automatic evaluation (details in Section F.2).
Results are in Table (top half). Baseline outputs do not conform to input object pose due to the lack of precision of text prompts, and may fail to preserve input background as observed in Fig. 2.
Ablation on Conditional Sampling, Section 3.1 introduced the conditionally sampling strategy, which is crucial for efficient sampling from the product distribution. Removing the updates in Eq. (6) ("No Cond" in Table 1) decreases performance and results in less harmonized visual outputs (Fig. 2).
Character Insertion. The task spec above closely relevant to the character insertion task studied in Magic Insert (Ruiz al., 2024), where the goal is to insert character from an image into background image. We use 10 background images paired with character inputs released on their official demo page construct evaluation dataset of 80 scenes. Since their model uses different

---

Published as a conference paper at ICLR 2026
Input Physical Simulation (4 keyframes)
Input Image
"Pigeons startled as the white cloth slips off the box and flutters to the pavement.
Input Image
"Cloth slips off the wooden box as a distant car emerges, its low rumble stürring dust."
Input Physical Simulation (4 key frames)
Input Image
"Raindrops burst into splashes. The spray in the foreground sinks pressure fades.
Input Image
"A bike rider speeds along the wet pavement water splashes from the wheels.
Figure 3: Application on Leal-Simullaiio-nnsttueee
Video Generation. Given an input
image and physical simulator rendering describing precise object motions, method generates videos aligned with input motions while synthesizing natural content for non-foreground regions.
Controllability
Video Quality
Semantic Alignment
Methods
(fg) (?)
(L)
GPT-4a
Smooth (1)
Aesthetic (1) Imaging (f)
VICLIP (1)
GPT 4o
Object-Centric Simelation Input
Traj2V 0.602 193 0.109 .333
0.6381.112
0.993
0.5314.077 071 0.570 154 0.725 179 0. 0.2581.044
0.757:1.11
Depih2V 0. 0.787+ 0.1034.00
0.650+.144
0,9814.00
0.581
0.750+.126
0.2614.041
0.775-..150
Image2V 2.9 0.111#.021
0. 2.7082.100
104 0.978 025 0.553+.057
0.611#.101
0.255+.036 .99
0.788=.079
Ours
0.730+.221 0.104 .30 0. 0.7082.100 0.9942 008 0.5494.068
0.625 0.842..
0.270=
038 0.8171.050
Full Scene Simulation lupuc (PhysGen3D (Chen et al., 2025) datasel)
PhysGi3
0.438 .99 0.995 00666-00220000 0.5714.114
0.673 048
0.5134.117 117 0.2244.028
0.588
0.2374.084
0.2634.122
0.9934 002
0.468:.097
0.410=.144
0.250+.20
0.2024.038
0.400
Depth2V
0.161#.065 .065 0.550
0.993
0.57941067
0.680 081
0.662 216 0.242 000
0.788
Image2V
0.242#.077 .77 0.525 192
1.986$t.010
0.594
0.662#.06 062 0.6384.24.
243 0.23644.022
0.78% 106
Ours
0.136110477 0.587 0.993
0.575$.071
0.6881.045 O. 0.7 0.23944036
0.825::.11
Table 2: Physies-Simulator-nsiiii
Video Generation Evaluation.
backbone (Podell et al., 2023), for fair comparisons, we also include three baselines using the same FLUX backbone as ours: Add-it (Tewel et al., 2024), SDEdit (Meng et al., 2021), and FI.UX-Fil.
Results are in Table (bottom half). Add-it receives text instructions for object insertion, but tends not to follow the specified character location in the prompts; SDEdit produces harmonized outputs but does not closely preserve the background due to the global noising operation; the inpainting method, FLUX-Fill, sometimes ignores the character descriptions in the input text instruction and fails to insert any object. These observations are also reflected qualitative samples in Section E.2.
4.2 PHYSICAL-SIMULAATA INSTRUCTED VIDEO GENERATION
Task. The setup from Section 4.1 can be directly extended to dynamic scenes for video genera- tion, with object poses and dynamics (e.g., ball bouncing, Fig. 3 top row) specified by physical simulator, an input image, and textual scene description. Below, we consider two sets of experts: flow-hased (Wang et al., 2025) and autoregressive-basee (Zhang & Agrawala, 2025).
7

---

Published as a conference paper at ICLR 2026
Inpur Sim
Ours
Traj2V Image
Frame Eo "Passing taxis, umbrella-clan crovdss, flickriing billboadds...e as. a metal sphere falls down."
Frame f
Frame l2
Frame iy
Frame E2
Frame lg
seagult stands in a puddle. Other gulls mill about behind, while a ball sails past a seagull." "
Video Generation. Predictions are pro- cessed in grayscale and overlaid with estimated tracking (Xiao et al., 2024) for visualization.
Figure 4: Comparisons on
Flow-Based Experts. We evaluate our method and several monolithic models, including image-to- video (Image2V), depth-to-video (Depth2V), and trajectory-to-viiee (Traj2V), all using the same 1 (14B) model as backbone, on a dataset consisting of simulation inputs paired with 4 input images each, giving a total of 12 scenes with examples in Fig. 3. To obtain the initial frame for all methods, we edit the input image with our method described in Section 4.1 to insert the simulated object. Depth maps required as method inputs are rendered from the physical simulator, and input trajectory maps are computed using object centroids. Metrics include the ones from Section 4.1 that are relevant to this task, with additional ones adopted from a video benchmark VBench (Huang et al., 2024): motion smoothness scores introduced in VBench, MUSIQ (Ke et al., 2021) for imaging quality, and ViCLIP (Wang et al., 2023) for text prompts alignment. We further compute foreground motion trajectory accuracy with IoU on SAM2 (Ravi al., 2024) detected from output videos.
Results are in Table 2 (top half). Depth-to-vides and trajectory-to-video alone fail to capture rich moving and birds running in Fig. 4. In particular, they
non-foreground-objee motions, such tend to compensate for the object's downward motion with upward camera motions, as visualized by the tracking trajectories in Fig. 4. Image-to-video model does not follow the input motion.
Autoregressive Experts. We implement the autoregressive construction for Eq. (3) using two gener- ative experts: a fext--frmm-seetioo video prediction model, FramePack (Zhang & Agrawala, 2025), and a Gaussian distribution (x)) centered on physics simulator renderings. Denote video quence of the physics simulator's ' RGB rendering overlayed with the input initial frame as cm then (*) cxp (-wljx e- [[2) with constant E R. In this case, Z1:t represents the first T frame sections of a video x. We approximate sampling from this distribution with its MAP solution, which amounts gradient updates w.r.t. loss [im I| Qualitative results are in Fig. 5, suggesting that composing per-expert constraints enforces desired object motion on top of output photorealism.
Full-Scene Simulation. In the above setting, simulation inputs only describe foreground objects. We further evaluate cases where full-scene simulation is available. We use the simulation results from 9 released scenes from PhysGen3D (Chen et al., 2025), method that reconstructs and animates images in physical simulators, evaluation inputs. To better preserve scene content, we the flow-inverted noise vector for particle initialization as opposed to random noise, and compare with this inversion baseline. Results are included in Table 2 with qualitative samples in Fig. 8.
4.3 TEXT-TO-IMAGE GENERATION WITH LAYOUT CONTROL
Task and Evaluation. This task aims to generate an image given an input global text prompt and set of object bounding boxes with paired object text descriptions. We evaluate on 50 scenes ran-
8

---

Published as a conference paper at ICLR 2026
Simulation
Ours
FramePack
"A dog watching a red ball falling and bounces off."
Ours
FramePack
bird dancing while a white rubber ball is falling down."
Frame 4
Frame &
Frame ty
Frame t
Frame tq
Figure 5: Physical- Simuultion-Inntruut Video Generation compared with the baseline with the same backbone. Our method better adheres the input object motion trajectory.
Method
mloU (f)
VQAScore (f)
VQAScore-R (f)
3DIS-FLUX
0.673:10.182
0.800
0.808 161
L 1 (Du atal. (2023))
0.675=40.200
0.567++0.272
0.367 189
0.695220.172
0.802:10.194
0.795
L 8
0.715=0.1177
0.879=#0.115
0.8/13#001335
0.723=0.176
0.897-40.106
0.870+0.111
1. 32
0. 0.72820.11
0.904:0.092
0.881 110115
Table 3: Text-to-Image Generation with Layout Controls on MIG-Bench (Zhou et al., 2024) dataset, comparing our method with varying compute budgets and an application-specifc bascline.
domly sampled from the full MIG-Bench (Zhou et al., 2024) dataset of 800 scenes. Metrics include bounding box mloU, detected using GroundingDINO (Liu al., 2024b), and VQAScore (Lin al., 2024) measuring output alignment with global and regional prompts.
Results. Table 3 and Fig. contains results on ablating the number of particles L and also com- parisons with 3DIS-FLUX (Zhou et al., 2025), state-of-the-art method designed for text-to-image generation with the same backbone model as ours. the case of = 1, the method reduces to Du al. (2023). The performance of method improves with higher computation budgets, and alleviates the need for application-speeifee designs, such as cross-attention layer intervention as used in Zhou al. (2025). The ablation SMC resampling deferred to Section E.5.
5 CONCLUSION
We have introduced a probabilistic framework where knowledge from heterogeneous sources is composed in the form of product distributions, and proposed an efficient sampling algorithm in- terleaving annealed MCMC sampling and SMC resampling to integrate guidance from multiple generative and discriminative models. Our training- free approach can effectively integrate visual generative priors, VLM guidance, and physics-based constraints. Empirical evaluation suggests that this method generates images and videos with improved controllability and fidelity compared to prior works, providing a practical recipe for controllable visual synthesis tasks.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is in part supported by the Stanford Institute for Human-Centered AI (HAI), AFOSR YIP FA9550-23-1-42 23-1-0177. ONR N00014-23-1- 1-235-,,,,, ONR YIP N-0014--4 ONR MURI N00014- 22-1-2740, and NSF RI #2211258 and #2338203.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303, 087744 2023. 1
Michael Bacharach. Scientific disagreement. Unpublished manuscript, 1972.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint UrXiv:2309.16600 2023.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In CVPR, pp. 843-852, 2023. 2
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusiin: Fusing diffusion paths for
controlled image generation. arXiv preprint arXiv:2302.08113, 2023. 2
Yuri Boykov and Gareth Funka-Lea. Graph cuts and efficient nd image segmentation. International journal of computer vision, 1091)11 2006.
Yuri Y Boykov and M-P Jolly. Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In Proceedings eighth IEEE international conference on computer vision. ICCV 2001, volume pp. 105-112. IEEE, 2001.
Arwen Bradley. Preetum Nakkiran, David Berthelot, James Thortoo,, and Joshua M Susskind. Mechanisms of projective composition of diffusion models. arXiv preprint arXiv:250..04549, 2025. 2
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators, 2024. URL pettps://opeatti com/ research/
models asse-aa--- as worlwordd--o simulators.
Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, et al. Go-with-the--ie Motion-controllable video diffusion models using real-time warped noise. arXiv preprint wxXiv2250..0333 2025. 2
Boyuan Chen, Hanxiao Jiang, Shaowei Liu, Saurabh Gupta, Yunzhu Li, Hao Zhao, and Shenlong Wang. Physgen3d: Crafting miniature interactive world from single image. CVPR, 2025. 2, 7, 8, 20
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209. 14687, 2022.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Advances Neural Information Processing Systems, 2021. 2, 21
Arnaud Doucet, Nando De Freitas, and Neil Gordon. An introduction to sequential monte carlo methods. Sequential Monte Carlo methods in practice, pp. 3-14, 2001a.
Arnaud Doucet, Nando De Freitas, Neil James Gordon, et al. Sequential Monte Carlo methods in practice. Springer, 2001b.
Yilun Du and Leslie Kaelbling, Compositional generative modeling: A single model not all you need. arXiv preprint arXiv:2402.011003 2024. 2,3
10

---

Published as a conference paper at ICLR 2026
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based mod- els. Advances in Neural Information Processing Systems, 35:6637-6647, 2020. 1,2,
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In ICLR, pp. 8489- 8510. PMLR, 2023. 2,
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Associa- tion, 106(496):1002-166) 1614, 2011. 5
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024. 16
Leonhard Fuler. Institutiones calculi integralis, volume 1. impensis Academiae imperialis scien- tiarum, 1792. 16
FLUX. Flux, http:p/s///tttta...o/o///eeeettt
2024. 6
Timur Garipov, Sebastiaan De Peuter, Ge Yang. Vikas Garg, Samuel Kaski, and Tommi Jaakkola.
Compositional sculpting of iterative generative processes. arXiv preprini arXiv:2309.16115.
2023. 2
Sachit Gaudi, Gautam Sreekumar, and Vishnu Boddeti. Coind: Enabling logical compositions in diffusion models. arXiv preprint arXiv:2503.00 145, 2025. 2
Tomas Geffner, George Papamakarios, and Andriy Mnih. Compositional score modeling for simulation- inference. In International Conference on Machine Learning, pp. 11098-11116. PMLR, 2023.
Christian Genest and James V Zidek. Combining probability distributions: A critique and an anno- tated bibliography. Statistical Science, 1(1):114-135, 1986.
Ben Glocker, Nikos Komodakis, Georgios Tziritas, Nassir Navab, and Nikos Paragios. Dense image registration through mrfs and efficient linear programming. Medical image analysis, 12(6):731- 741, 2008.
Will Grathwohl, Kuan-Chieh Wang, Jömn-Herik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03250 0922992....3....... 2019. 2
Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, Wenping Wang, and Yuan Liu. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. arXiv preprint arXiv:2501. 033447, 2025.
Jiajun He, José Miguel Hernández-Lobato, Yuanqi Du, and Francisco Vargas. Rne: plug-and-play diffusion inference-time control and energy- based training, 2025. URL https: /arxiv.
org/abs/ 2506. 05668. 2
Xuming He, Richard S Zemel, and Miguel A Carreira-Perpinan. Multiscale conditional random fields for image labeling. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., volume 2, pp. II-II. IEEE, 2004.
Yutong He, Naoki Murata, Chich-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. arXiv preprint arXiv:2311. 2023. 2
Geoffrey E Hinton. Products of experts. International Conference Artificial Neural Networks, 1999. 1
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 1478):177111800, 2002.

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
2, 3, 16
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02250
2022.
Xun Huang, Arun Mallya, Ting- Chg--Chun Wang, and Ming- Yu Liu. Multimodal conditional image synthesis with product-of-expers gans. In ECCV, pp. 91-109. Springer, 2022. 1,2
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprchensive benchmark suite for video generative models. In CVPR, pp. 21807-21818, 2024.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 20
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. ICCV, pp. 5148-5157, 2021. 8
Vladimir Kolmogorov and Ramin Zabih. Multi-camera scene reconstruction via graph cuts. In European conference computer vision, pp. 82-96. Springer, 2002.
Sanjiv Kumar and Martial Hebert. Discriminative fields for modeling spatial dependencies in natural images. Advances in neural information processing systems, 16, 2003.
Hugo Larochelle and lain Murray. The neural autoregressive distribution estimator. In Proceed- ings of the fourteenth international conference on artificial intelligence and statistics, pp. 29-37.
JMLR Workshop and Conference Proceedings, 2011. 3
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image
pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Shuang Li, Yilun Du, Joshua B Tenenbaum, Antonio Torrllb,, and Igor Mordatch. Composing ensembles of pre-trained models via iterative consensus. arXiv preprint arXiv:2210.115221 2022.
2
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pp. 366-384. Springer, 2024. 9, 19
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXin:2210.02747, 2022. 3
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez- Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:2412.0226 2024. 16
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In ECCV, pp. 423-439, Springer, 2022a.
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics- grounded image-to-vin to-veooo generation. In European Conference on Computer Vision, pp. 360-378. Springer, 2024a. 2
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference computer vision, pp. 38-55. Springer, 2024b.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022b. 3
12

---

Published as a conference paper at ICLR 2026
Grace Luo, Jonathan Granskog, Aleksander Holynski, and Trevor Darrell. Dual-process image generation. arXiv preprint arXiv:2506.01990 2025.
Divyat Mahajan, Mohammad Pezeshki, Ioannis Mitliagkas, Kartik Ahuja, and Pascal Vincent. Com- positional risk minimization. arXiv preprint arXiv:2410.06300 2024.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073: 2021." 7
Antonio Montanaro, Luca Savant Aira, Emanuele Aiello, Diego Valsesia, and Enrico Magli. Mo- tioncraft: Physics-based zero-shot video generation. Advances in Neural Information Processing Systems, 37:123155-12316 2024. 2
Radford M Neal. Annealed importance sampling. Statistics and Computing, 11(2):125-139, 2001. 1,3
Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. arXiv preprint arXiv: 1601 .06759, 2016. 3
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023."
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021."
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Fe- ichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. URL https: : / arxiv. org/ / abs / 2200..00 00714. 8, 20
Christian P Robert, George Casella, and George Casella. Monte Carlo statistical methods, volume 2. Springer, 1999.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022. 1
Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Semantic image inversion and editing using rectified stochastic differential equations. In ICLR, 2025.
Nataniel Ruiz, Yuanzhen Li, Neal Wadhwa, Yael Pritch, Michael Rubinstein, David Jacobs, and Shlomi Fruchter. Magic insert: Style-aware drag-and-dropp arXiv preprint arXiv:2407.02489, 2024. 6
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- tion Processing Systems, 15:36479-364999 2022. 1
Christoph Schuhmann and Romain Beaumont. Laion-aesthetics. LAION. AI, 2022.
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time sealing and steering of diffusion models. arXiv preprint arXiv:2501.06846 2025.
Marta Skreta, Tara Sahound-Sadegh, Viktor Ohanesian, Roberto Bondesan, Alán Aspuru-Guzik, Arnaud Doucet, Rob Brekelmans, Alexander Tong, and Kirill Neklyudov. Feymman- kac correc- tors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819
2025. 2
13

---

Published as a conference paper at ICLR 2026
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265, pmlr, 2015. 3
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for blug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483-32498. PMLR, 2023.
Jocelin Su, Nan Liu, Yanbo Wang, Joshua Tenenbaum, and Yilun Du Compositional image decomposition with diffusion models. arXiv preprint arXiv:2406. 19298, 2024. 2
Xiyang Tan, Ying Jiang, Xuan Li, Zeshun Zong, Tianyi Xie, Yin Yang, and Chenfanfu Jiang. Phys-
motion: Physics-grounder dynamics from a single image. arXiv preprint arXiv:2411.17189, 2024.2 2
Yoad Tewel, Rinon Gal, Dvir Samuel, Yuval Atzmon, Lior Wolf, and Gal Chechik. Add- it: Training-fre free object insertion in images with pretrained diffusion models. arXiv preprint arXiv:2411.07232. 2024. 7
James Thornton, Louis Bethune, Ruixiang Zhang, Arwen Bradley, Prectum Nakkiran, and Shuangfei Zhai. Composition and control with distilled energy diffusion models and sequential monte carlo. arXiv preprint arXiv:2502.127788 2025.
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng. Jiayu Wang. Jingfeng Zhang. Jingren Zhou, Jinkai Wang. Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pan- deng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang. Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint ar WwI.:i5533203111111 2025.
Chaohui Wang. Nikos Komodakis, and Nikos Paragios. Markov random field modeling, inference & learning in computer vision & image understanding: survey. Computer Vision and Image Understanding, 117(1 (1(11))1610-1621 2013. 1
Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint 2rX44:2411.04746, 2024. 6
Yi Wang. Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understand- ing and generation. arXiv preprint arXiv:2307.06944 2023. 8
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning (ICML-I1), pp. 681 681-688, 2011. 3
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. NeurlPS, 36:31372-331444 2023.
Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng. Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. In CVPR, 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In NeurlPS, pp. 15903-15935, 2023.
Sherry Yang, Yilun Du, Bo Dai, Dale Schuurmans, Joshua B. Tenenbaum, and Pieter Abbeel. Prob- abilistic adaptation of black-box text-to-video models. In ICLR, 2024. 2
14

---

Published as a conference paper at ICLR 2026
Xindi Yang, Baolu Li, Yiming Zhang. Zhenfei Yin, Lei Bai, Liqian Ma, Zhiyong Wang, Jianfei Cai, Tien-Tsin Wong, Huchuan Lu, et al. Towards physically plausible video generation via vlm planning, arXiv preprint 250323280,,,,368, 2025.
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Y Zou, and Stefano Ermon. Tfg: Unified training-free guidance for diffusion models. NeurlPS, 37: 12370-22411 2024. 2
Lvmin Zhang and Maneesh Agrawala. Packing input frame contexts in next-frame prediction models for video generation. Arxiv, 2025. 7,
Qinsheng Zhang, Jiaming Song. Xun Huang, Yongxin Chen, and Ming- Yu Liu. Diffcollage: Parallel generation large content with diffusion models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10188- 10198. IEEE, 2023. 2
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectivvness of deep features as perceptual metric. In CVPR, 2018. 6
Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Grosse. Probabilistic inference in language models via twisted sequential monte carlo. arXiv preprint arXiv:2404.17546, 2024.
Dewei Zhou, You Li, Fan Ma, Xiaoting Zhang, and Yi Yang. Mige: Multi-instance generation controller for text-to-image synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6818 18-88828, 2024.
Dewei Zhou, Ji Xie, Zongxin Yang, and Yi Yang. 3dis-flux: simple and efficient multi-instance generation with dit rendering. arXiv preprint 2xXhv:2501.05131, 2025. 9,
15

---

Published as a conference paper at ICLR 2026
A OVERVIEW
The appendix contains discussions (Section B), extended descriptions the framework (Section C), and additional experimental results (Section E) and details (Section F).
B DISCUSSIONS
Compute Requirement. All image experiments are run using NVIDIA H200 GPUs and all video experiments with 2 NVIDIA H200 GPUs. Generating one sample in Section 4.1 takes approximately 4min. It takes 5min for flow-based and 30min for sutoregressive-basee models in Section 4.2. Wall- clock time for the task in Section 4.3 with different numbers of particles is included in Table 5.
Limitations. The proposed method requires extra computation due intermediate MCMC steps and parallel sampling compared to vanilla feedforward inference the image or video genera- tive model backbones. Furthermore, the sampling efficacy relies the assumption that expert predictions are reasonably compatible, and strong incompatibilities might require other sampling techniques or training. Qualitative examples are discussed in Section E.4.
This work proposes a framework and provides empirical validation with small number of experts and we leave large-scale sampling as future work. We mainly highlight two straightforward sources for parallelism: MCMC sampling across particles, and inference for different generative experts within each MCMC step evaluation.
Societal Impact. We believe outputs of the proposed framework does not directly possess negative societal impact. The underlying technology enable more content creation tools and benefits under- standing model behaviors. However, highly controllable generation could be exploited misinformation While our work does not inherently use sensitive data, we are aware of such risks and closely follow ethical guidelines the community to help mitigate these risks.
C FRAMEWORK DETAILS
C.I FLOW MODELS
We use bar notations for pre-trained models in the following discussions.
Consider the i-th generative expert, which is a flow model that defines a time-dependent continuous probability path (50) (z; f), Rd [o, 1], with boundary conditions ((0) (z:; 0) = A/ I) and
(i) g (z; 1) Paata (x). Each flow model associated with a scheduler ā, ō [0, 1] + E that defines (6)
conditional probability path (x) y; N(x ā(Dy, (())), y Paam (v), and is trained to predict velocity field (x; f, 0,) that generates the marginalized path gf f), where 0 denotes model parameters. While the training-tine time scheduler (ā. o) can differ across experts, they can be aligned during inference time via scale-time transformation (Lipman et al., 2024).
Given a time remapping function & (T,T - 1, ... }} + [0, 1}. one can construct the per-expert product component from Eq. (3) via discretizing the path p( (x; 2) into
(-)))=='))() (z:
Pr f (x)= (O, 7) is easy to sample from, and (z) Pesy (x) encapsulates the prior data distribution of the expert model,
We require that { is monotonic and {(7) = (1) = 1. By construction,
fulling motivation in Section 3.1. For all experiments, we define to be the Euler discretiza- tion (Esser et al., 2024; Euler, 1792), with additional alignment to the DDPM scheduler (Ho et al., 2020) for experiments in Section 4.3.
16

---

Published as a conference paper at ICLR 2026
Line 5 from Algorithm is computed as follows:
( (x) ((;
1/2
0g = a({(t)),
dt (E(t)),
8 (x) -ani (z) + ăx
(0)
0,0 à
(x) = Σ (c)o
sx(x) Σ (z)s
Keot+1 [x) - 8(x' (x + (x) -) - E(I))))
1G x) = Naa'; x²
2 (x), 121).
where (xx) is defined in Section F.1, with all entries of Σ; (x) to b.
The end-point prediction is available
5
(x())
âr - 8,0% Wr.
- 8,00
C.2 FRAMEWORK INSTANTIATION WITH AUTOREGRESSIVE EXPERTS
[YZ: TODO]
C.3 APPROXIMATION FOR CONDITIONAL SAMPLING
We explain Eq. (6) below. Let xy be any expert. We approximate the distribution p(r; (pp(i)) as
p(z: TPa(t) x p(a)))((T)() x.) p(x,)
II #Epa(s)
p( xx),
(9)
where parent regions zy are assumed to be conditionally independent given wi. model each conditional distribution p(ay x.), we define Gaussian distribution of the deviation of the flow vector predicted at Fy and x,, where p(z, xx.) x -=]]((( with constant w R. Under this parametric distribution, when and 2; have consistent flow predictions, the likelihood is high (which is reasonable as this indicates both 24 and x; are mutually compatible). We can convert the probability expression in Eq. (9) to a corresponding score function, resulting in Eq. (6).
In this work, pa(i) defined as the generative expert defining natural data distribution for global E Idd, e.g., coxtioonditioned generative model for the full image/video æ. Each generative expert is additionally conditioned framework allows for different types of conditions across experts, enabling flexible control handles that are typically application-depender.
context signal c, Rd e.g., texts or depth maps. Note that the
D CORRECTNESS
D.1 RELATION OF PRODUCT DISTRIBUTION AND JULTI-CONDITIONER SAMPLING
Conditional independence; require sum of expert weights to be
D.2 SAMPLING UNBIASEDNESS
[YZ: TODO]
17

---

Published as a conference paper at ICLR 2026
Expert I data
Expert
data
(2) Product datal
data data
Figure 6: Mixture-of-Gausiann Data Visualization.
WI (4)
(4)
(L)
TV (L)
NL.L (4)
Expert 1 (p (4))
0.906
1.033
0.047
0.709
7.727
Expeit 2 (p (2)
0.883
1.023
0.045
0.702
8.197
No-amnealing (I' = 1)
3.765
4.005
5.075
0.999
21.129
No MCMC (K
0.728
0.909
0.027
0.604
5,008
Ours
0.580
0.777
0.018
0.583
4.517
Table 4: Comparisons on Mixture-of-Gausiann Data.
E EXTENDED EXPERIMENTAL RESULTS
E. SYNTHETIC EXPERIMENTS
We conducted experiments using a synthetic mixture-of-GGassiaans dataset with known ground-truth product distributions for further quantitative evaluation.
Dataset. We use 2D mixture-of-Gaausian as the target distribution for two experts Paata and Paag The first expert mixture consists of Gaussian whose means are equally spaced along the r-axis with anisotropic covariance; the second is defined analogously along the y-axis. Fig. 6 shows the visualization of ground truth distributions.
Metrics. WI and W2 are empirical 1- and 2- Wassersttei distances between method samples and ground truth ones. MMD is the squared maximum mean discrepancy with RBF kernels. TV is total variation distance. NLL is the Monte Carlo estimate of the negative likeliiel -E [log p data (r) of method samples under the analytic density (x))..
Models. Each expert model 5-layer MLP with hidden dimension 512. Models are trained with flow matching for 10000 iterations with Adam (?) optimizer and learning rate le 3. During inference, for each method, we choose the number of discretization steps T and the number of Langevin steps K so that the number of function evaluation (NFE) is constant across models. NFE computed with (K + I) - 1) since each step involves 1 query for initialization and K for MCMC updates, and there are T - steps in the for loop in Algorithm 1.
Results. Results are reported in Table 4. Our method achieves better sampling quality compared to all baselines, indicating improved mixing on this dataset, with sharply peaked energy surface of the target product distribution.
E.2 GRAPIICS-ENGIN INSTRUCTED IMAGE EDITING
Fig. 7 contains qualitative samples on the Magic Insert dataset for experiments in Section 4.1.
E.3 PHYSICAL-SIMULATTO CANNIALL-SSIILLLLO--ATTITTTEEEET INSTRUCTED VIDEO GENERATION
Qualitative samples included in Fig. 8.
18

---

Published as a conference paper at ICLR 2026
Method
mloU (1)
VQAScore (1)
VQAScoreR (7)
Wall-Clock Time (minn)
0.675=0.200
0.567=0.272
0.56720.188
L 1 (No SMC)
0.675=0.200 200 (+00.000)
0.567+0.272 (+000000
0.567+0.1188 (++00000)
~
0.695=0.172
0.802#0.190
0.795=0.155
L 4 (No SMC)
0.715=0.170 (4-0.020)
0.796=0.117 (-0.007)
0. 777720.155 (-0.028)
0.715:10.177
0.879:10.115
0.813:10.
~6
L (No SMC)
0.735=0.116 (+00000)
0.807
(-0.072)
0. 797790.111 (-
~6
L 16
0.723
0.897=0.100
0.870=#0.122
~=1
16 (No SMC)
0.698=0.175 (-0.025)
0.81 166 (-0 0.033)
0.80622.166 (-0.064)
~1
L = 32
0.728 100117
0.90110.092
0.88110.118
~25
32 (No SMC)
0. =0.110.1133 (-0.037)
0.85420. 139 (-0.550
0. 0.83840.136 (-0.43)
~22
Table 5: SMC resampling ablations on the task from Section 4.3. Values in green (red) indicate "No SMC" is better (worse). Results suggest that SMC benefits sampling efficiency. Resampling has relatively marginal computation.
E,4 TEXT- TO-TTGO--MAE GENERATION WITH REGIONAL COMPOSITION
Fig. 9 contains qualitative samples on the MIG-Bench dataset for experiments in Section 4.3,
This task exemplifies challenges when multiple expert models (one for each image region) are com- posed, and we observe two failure modes expanding the discussions from Section B.
The first arises when input conditions are contradictory. An example is shown the second-to-last row of Fig. 9, where the regions for "a yellow horse" and "a blue horse" overlap. In this case, the support of the product distribution becomes very small, and although our method draws samples from the product, the resulting samples may not be visually desirable.
The second occurs when some input conditions are complex and our method inherits limitations of the underlying expert models. For example, in the last row of Fig. 9, the text prompt for the global region specifies attributes for multiple objects, which leads to attribute leakage: the global expert makes all three hot dogs blue in the result for L == 32.
E.5 ABLATION ON SMC
We conduct ablation for SMC with results shown in Table 5,
E.6 TEXT-TO-IMAGE GENERATION
We further demonstrate the efficacy of distribution composition the task of text-to-image gen- eration, where we first query an LLM to parse an input text prompt into spatial bounding boxes with regional prompts (Fig. 10 left), aiming to compose the distributions from text-to- text-to-image model (FLUX.1 [dev]) conditioned on regional text prompts, one for each region, as generative experts, and from VQAScore (Lin al., 2024) as discriminative expert, which gives scalar value predicting the probability that its input image is aligned with the input text prompt. We compare our methods using 16, 4 particles and 1 partiele (which amounts not using the discriminative expert), variant using no generative composition (equivalent to best-of-N sampling for vanilla FLUX) while matching the compute budget for 16 particles, and with the vanilla FLUX backbone. In the examples shown in Fig. 10, our compositional approach achieves better text alignment compared to the base- lines, and the discriminative expert provides crucial knowledge, e.g., counting, to steer the output distribution.
The parallel sampling process is visualized Fig. 11. For each scene from Section 4.3, we use an LLM to convert a text prompt into layout of bounding boxes following the procedure described in Section E.5. The input text prompts and resulting layouts and regional prompts are visualized in Fig. 10. Then, we follow Algorithm and initialize L 4 particles and annealed schedule length T = 28, which are reweighed at three key timesteps = 21, 19, 1 using the discriminative expert, VQAScore (Lin et al., 2024), which measures image-text alignment with probability score. At t 21, 19, we binarize the scores with the threshold set to be the median score of the current particles, discard the two below the threshold, and duplicate the remaining two high-score particles
19

---

Published as a conference paper at ICLR 2026
two times to maintain particles in each round. At t 1, we keep the highest-scoring particle as the final output sample.
Fig. 11 shows end end-point predictions to visualize the intermediate samples from the annealing process from t = 28 to t = VQAScore measurements (higher means more aligned) are overlayed on the top-left corner for each sample. In the proposed framework, integrating multiple generative experts allows for composing simpler regional prompts, construction of the annealed distributions allows for iterative refinement of samples, and parallel sampling further integrates the knowledge from the discriminative expert and improves output accuracy, as evidenced in the comparisons from Fig. 10.
F EXPERIMENT DETAILS
F.1 IMPLEMENTATION DETAILS
As described in Eq. (4), a generative expert can be defined over the entire scene æ E IRd or a region Ty E Rd with d( < d, in which case the expert has prior over only part of the scene, c.g., spatial region in Section 4.3, In these cases, we pero-pad the model prediction from Rd Rd to Rd.
In other cases, one may restrict the influence of a generative expert to update only a scene region, when the expert is defined over the full scene TRd e.g., when the expert only has knowledge over foreground object informed by the graphics engine or physics simulators (Sections 4.1 and 4.2). We apply regional (spatial for images and spatio-temporal for videos) weight A (xx) [0, 1J on the velocity prediction Ui (z) in these cases, where ( (x) is a coussian-blurred foreground mask.
F.2 AUTOMATIC EVALUATION WITH VLMS
We follow the protocol proposed in PhysGen3D (Chen et al., 2025) to run automatic evaluation for Sections 4.2 and 4.3 using GPT-4o (Hurst et al., 2024). For each scene, all methods' output images, uniformly sampled 10 frames when outputs are videos, are sent to GPT-4o together with template prompts as specified in PhysGen3D (Chen et al., 2025). GPT-4o is asked to give scores from 0 to for each video on all metrics. The scores for the same metric are averaged across all scenes.
F.3
INSTRUUT- IMAGE EDITING
In Section 4.1, the product distribution is defined as p(x) x pdepth2image (x: d ddapth (alepth)pimmee ) (x dext eimage j. We use T = 28, L 1, with 1 MCMC sampling step and 2 conditional sampling updates (Eq. (6)) per iteration. The spatial resolution is 960 x 1664 for the graphics engine dataset and 1024 1024 for Magic Insert dataset.
For the graphics engine dataset, assets are rendered in 2
SAM2 (Ravi al., 2024) to obtain the segmentation masks of foreground characters, and feed the images segmented characters, resized to the bottom- left quadrant and overlaid with backgrounds, into GPT 04-mini for image captions to serve text prompts for evaluated methods. Magic Insert results are downloaded from the official demo page. Add-it receives background images from the dataset as inputs, together with the GPT-generated captions (of overlaid images) appended with "(character) is at the bottom left quarter" to indicate the desired locations. SDEdit takes in input overlaid images and adds Gaussian noise with standard deviation 0.5 before denoising.
For Magic Insert data preprocessing,
F.4
CELLLLTORR-ISTRCTTEEE INSTRUCTED VIDEO GENERATION
In Section 4.2, for flow models, the product distribution defined as p(x) x depphh2video (a clext edepth ) imgggevvddeo iteration. We use no conditional sampling updates for this task due to GPU memory constraint. Each generated video has 81 frames and spatial resolution 480 832.
ddext cimage), ). We use T 30, L 1 with MCMC sampling step per
httpss: // thub. com coom/Geneee Genesi Rmbod ed- A /Genesis
20

---

Published as a conference paper at ICLR 2026
For autoregressive models, the product distribution is defined as p(z) & p cam "mmeeeneeevvele (æ clest, clmmage) with p= caim) defined Section 4.2. We approximate sampling from this dis- tribution with first sampling from p pimage2viden, then do 8 gradient updates with respect to La loss || c esimulation I Results in Fig. use 81 frames per sequence and spatial resolution 512 x 768. We set I = with 1 MCMC sampling step per iteration for this experiment.
F.5 TEXT-TO- IMAGE GENERATION WITH REGIONAL CONTROL
For Section 4.3, the product distribution is p(x) x (II; papthiimaee (Jix clexs (ecptt)) q g
(rr c
Depth), which predict scores for bounding- box-cropped images conditioning on regional text prompts and regional depth maps cropped from global depth maps predicted using the first stage of 3DIS-FLUX (Zhou et al., 2025) (layout-to-depth generation); y VLM discriminative expert VQAScore, where for each image sample, assigns the summed score computed over full image and its regions cropped with input bounding boxes as the reward. We use 28 and image resolution 1024 x 1024, with 1 MCMC sampling step and conditional sampling steps per iteration.
Here, plopthZinage is a regional generative experts
G EXTENDED DISCUSSIONS
G.1 GUIDANCE FOR DIFFUSION MODELS
Classifier guidance (CG) (Dhariwal Nichol, 2021) and classifier-fre free guidance (CFG) are canonical approaches for improving sampling quality of diffusion models. While our framework is a general sampler for product distributions and can be instantiated with diffusion- based and autore- gressive models, we proceed with the discussion fully within the context of diffusion models, and expand on the connection and differences of this work with CG and CFG below.
CG aims to sample from class candicoondi distribution by adding the score of base generative model and classifier trained on noisy inputs. However, because the product of diffused marginals does not equal the diffused marginal of the product distribution, CG results in biased scores. We formalize this claim in the follows.
Let P. be an operator density function f(z), E Rd, with
P.f(z)= = / N(xx; arto, f(o) dæo.
(10)
where ax, R are constants, then P.ʃ is the diffused marginal of f corresponding to noise level i as defined in standard diffuuson or flow-based models. In the case of CG for a fixed class c, the desired target distribution is p (z))
p(x))p'(c | 2;t) with unconditional distribution p(x), classifier p(c x; t). and gradient scale R. CG aims to sample from D following a Markov chain (G) 1220 Doing requires scores for P.p However, in general cases,
(zx) + (P:)) p(c w;;)²
(11)
and therefore the score of the right-hand side of Eq. (11) as computed in CG is biased for the true marginal Pt CG) which corresponds to the left-hand side.
Similarly, in CFG, for a fixed class and guidance scale s, p (2) := p' [ c)p'-*(z> (). For general densities u, v, (uu) # P.uPre. In particular,
Pap CPG Pt [p(- c)"p( 10) * (Pal- (ep(-10) (e)))))))PPPPppp 0))²~>
(12)
also leading to biased scores estimates.
On the other hand, when applied to sampling from D our proposed sampler explicitly sets the intermediate annealing target step t to the right-hand side of Eqs. (11) and (12) and uses additional MCMC transitions to asymptotically converge to the target, yiclding unbiased samples.
G.2 EVALUATION ON CASS-CONNDT SAMPLING
In the of class-conditioned sampling for pc and p CTG vanilla CG can be viewed as special case with K = 0 MCMC steps with K defined in Algorithm The advantages of setting > empirically verified in the following experiments.
21

---

Published as a conference paper at ICLR 2026
Method
W1 (
W22
MMD (4)
TV (J)
NL)
Vanilla K
0.581
0.724
0.389
0.833
2.796
CFG (s 2, K = 0)
0.296
0.476
0.026
0.671
2,631
K
Table 6: Evaluation on Class-Conditioned Sampling.
Baselines. We compare the following methods (s is guidance scale as defined in Section G.I, and K the number of MCMC steps): vanilla classcconiiii sampling without guidance with s 1,K = O; CFG with guidance scale with s = 2, K = O; and our sampler with additional MCMC K 0. All models are 5-layer MI.Ps with hidden dimension 512, trained
refinement with with flow matching. We zero out class conditioning with probability 0.5 during training. Similar to Section E.I, the step number T' for each method is determined to maintain constant NFEs; each step for CFG involves 2 NFEs while others involve 1.
Dataset and Metrics We use a synthetic 2D dataset forming an 8x8 checkerboard, where each column corresponds one class. Ground truth data visualization shown Fig. 12. Metries follow Section E.
Results. Results are reported in Table 6. Applying additional MCMC steps to the class-conditioned sampling tasks, with total NFE controlled, results in better sampling quality compared diffusion model inference with and without CFG.
22

---

Published as a conference paper at ICLR 2026
Overlayed Inputs
Magic Insert
Add-it
FLUX-Fill
SDEdit
Ours
Figure 7: Comparisons on Character Insertion Task.
23

---

Published as a conference paper at ICLR 2026
Input Sim
Ours
Depth2V
Imnge2V
Frame t
Frame Eg
Frame č3
Frame tą
Frame to
Figure 8: Baseline Comparisons PhysGen3D Data. Our method improves the visual quality compared to the input simulation and adheres more closely to the input object motions compared to baselines.
24

---

Published as a conference paper at ICLR 2026
Input Layout
3DIS-FLUX
(Du al. (2023))
4
32
32 No SMC
Figure 9: Comparisons on Layout-Conditioned Image Generation Task.
25

---

Published as a conference paper at ICLR 2026
an ant standing on green leuf directing, them
"five ants carrying
biscuats"
"Five ants are carrying biscuits, and an ant that is not currying biscuits is standing on green leuf directing them."
"one blue bloom field flowers where every bloom yellow"
"In a field flowers, every bloom is yellow, save for one that is blue.
Regional Layout
Ours (16 particles) Ours (4 particles)
No Disc. Expert
No Gen. Composition Vanilla FLUX
Figure 10: Text-to-Image Generation.
26

---

Published as a conference paper at ICLR 2026
0.449
0.291
0.590
0.472
0.565
0.531
0.698
==
0. 1869
0.9
0.914
0.899
t=1
Particle "Pive ants are earrying biscuits, and an ant that not carrying biscuits is standing green leaf directing them."
Particle
Particle 3
Particle 4
0.951
0.291
0.893
0.927
21
0.956
0.961
0,950
0.969
0.971
0.971
0.969
t=1
Particle
Particle
Particle
Particle 4
"In field of flowers, every bloom is yellow, save for one that is blue."
Visualizations of Parallel Sampling with Discriminative Experts. Discriminative expert (VQAScore) scores are annotated on the top-left corners; score underlining means the sample proceeds to the next annealed distributions (with smaller t). Images with the same boundary color share the same initial seed.
Figure 11:
27

---

Published as a conference paper at ICLR 2026
Ground Truth cond ddaaa
Figure 12: Class-Conditioned Data Visualization.
28


Reviews:
[{"review_id": "9mytWpNskT", "reviewer": "Reviewer_BVuA", "summary": "This paper aims to solve a critical challenge: given models with rich priors and knowledge over shared data domains, how to effectively integrate this diverse knowledge from multiple sources. The paper proposes an annealed MCMC sampler and SMC resampling for better model composition. The qualitative and visual results show success in both image and video generation.", "questions": "In Table 2, can the authors provide video quality metrics for analysis? It appears comparatively weak in Aesthetic compared with other baselines; a further explanation (visual or textual) would clarify the results.\nCan the authors also provide more failure cases where the framework does not perform well (in either image or video quality, efficiency, or controllability)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is clearly presented, with sophisticated structure and logical flow.\n2. It addresses an interesting and significant problem: how to effectively utilize different experts for efficient and controllable visual generation.\n3. The paper presents detailed experimental results, showing improvement compared to baselines.\n4. It also demonstrates strong visual results with clear corresponding explanations.", "weaknesses": "Method and motivation — Sections 3.1 and 3.2 emphasize that, to sample from the product of a set of generative experts, the paper uses Markov Chain Monte Carlo to iteratively refine samples based on their likelihood under the product distribution (3.1), and employs AIS and SMC to draw samples from the product-of-experts distribution. However, there still exists concern about the intrinsic sharpness of the product-of-experts energy landscape, even with intermediate tempered distributions and particle resampling weighted by discriminative likelihoods.\nThe overall energy surface can remain highly peaked. There is a chance that particles collapse into narrow regions of high probability, perhaps leading to a drop in effective sample size and poor mixing.\nCan the authors provide more evidence that the proposed framework overcomes the structural difficulty imposed by additive log-likelihoods in the PoE formulation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Ss3SXCWkA", "reviewer": "Reviewer_jLD4", "summary": "Modern neural models possess complementary knowledge about data like images and videos, but combining this diverse expertise—from generative AI to structured simulators—is still challenging. The authors proposed a probabilistic framework that unifies these heterogeneous models into a single product distribution. For controllable image/video synthesis, the paper introduces an efficient inference-time sampler using annealed MCMC with SMC-style resampling. The proposed method provides superior controllability and more flexible user interfaces than monolithic approaches.", "questions": "Please refer to weaknesses for my main concerns.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The proposed method achieves visually satisfactory generative results. The paper is well-structured and easy to follow.", "weaknesses": "The performance improvements yielded by the proposed approach is kind of small, compared to the benchmarking algorithms, such as Depth2V for the setting of Object-Centric Simulation Input.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wY73xaUWp1", "reviewer": "Reviewer_SGff", "summary": "The authors proposed a sampling method from a joint of generative models with potential discriminators. Their idea is to apply AIS on the product of generative distributions, then interleave the AIS steps with an SMC reweighting using the discriminators when they are presented. The authors evaluated their method both qualitatively and quantitatively on various visual modalities.", "questions": "1. The ablation study is a bit limited. How would the vanilla classifier-free guidance and classifier-based guidance perform with all these expert models? This should serve as the baseline for justifying the necessity of the proposed sampling scheme.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow.\n2. The proposed method is general and works well with different expert models across different modalities.\n3. The generated results can show better quality against the competing baseline, e.g., in Figure 2 the proposed method preserves the original image more faithfully.", "weaknesses": "1. The proposed method is more heuristic than it appears to be. The main novelty is applying some tweaks to well-established sampling methods when discriminators are involved, which work well on the modern large expert models. At some point, this feels like a glorified classifier-based guidance with no theoretical guarantee provided. Maybe the authors can tone down a bit.\n2. The model performance is not always on the better side. This includes both the metrics across the tables and the visual examination such as Figure 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8LmBXDUG6K", "reviewer": "Reviewer_WhsG", "summary": "This paper introduces a training-free, probabilistic framework for controllable visual generation by combining heterogeneous pre-trained \"expert\" models (e.g., generative models, discriminative VLMs, and physics simulator software). The core of the method is a novel sampling algorithm that draws from the combined \"Product of Experts\" distribution by interleaving three techniques: Annealed Importance Sampling (AIS) to gradually refine samples from noise, MCMC to ensure fidelity to the generative experts, and Sequential Monte Carlo (SMC) to resample and filter particles. The method is instantiated and evaluated on complex tasks, including graphics-engine-instructed image editing, physics-instructed video generation, and layout-controlled text-to-image synthesis. The results demonstrate superior controllability over baselines, effectively adhering to precise object poses and physics-based motion trajectories while maintaining high visual quality.", "questions": "Please see the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper provides a classic and mathematically sound algorithm for combining heterogeneous pre-trained models, including both generative and discriminative experts. The method can generate images and videos with high controllability without extra training.\n\n- The paper demonstrates many instantiations of this framework across different tasks (image editing, video generation, layout control) and expert types (flow models, autoregressive models, VLMs, physics engines), and the proposed method consistently outperforms monolithic baseline approaches.\n\n- The main paper and the appendix provide extensive implementation details, qualitative results, and ablations, which support its claims and reproducibility.", "weaknesses": "- The method is inherently slow due to its iterative nature, requiring $T$ annealing steps, $L$ parallel particles, and $K$ MCMC steps per particle. This makes it less practical for real-world GenAI applications, with image generation taking ~4 minutes and video generation taking 5-30 minutes. In my understanding, the proposed method is more like a proof of concept instead of a feasible path for future visual generation frameworks.\n\n- The use of the physics engine can be \"ad-hoc.\" For instance, the autoregressive video task requires the user to provide a full RGB rendering from the simulator. If a user must **manually** create complex 3D renderings or animations in professional software like Blender to serve as a control signal, the practical cost and effort may be prohibitively high.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)