Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation
Authors: Zijing Hu
Yunze Tong
Fengda Zhang
Junkun Yuan
Jun Xiao
Kun Kuang
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion Models
Alignment
Primary area: generative models
TL;DR: This work proposes asynchronous diffusion models, a novel framework that dynamically modulates pixel-wise denoising with varying timestep schedules, yielding improved text-to-image alignment.

Abstract: Diffusion models have achieved impressive results in generating high-quality images. Yet, they often struggle to faithfully align the generated images with the input prompts. This limitation is associated with synchronous denoising, where all pixels simultaneously evolve from random noise to clear images. As a result, during generation, the prompt-related regions can only reference the unrelated regions at the same noise level, failing to obtain clear context and ultimately impairing text-to-image alignment. To address this issue, we propose asynchronous diffusion models, a novel framework that allocates distinct timesteps to different pixels and reformulates the pixel-wise denoising process. By dynamically modulating the timestep schedules of individual pixels, prompt-related regions are denoised more gradually than unrelated regions, thereby allowing them to leverage clearer inter-pixel context. Consequently, these prompt-related regions achieve better alignment in the final images. Extensive experiments demonstrate that our asynchronous diffusion models can significantly improve text-to-image alignment across diverse prompts. The code repository for this work is available at https://github.com/hu-zijing/AsynDM.

Published as a conference paper at ICLR 2026
ASYNCHRONOUS DENOISING DIFFUSION MODELS FOR ALIGNING
GENERATION
Zijing Hu', Yunze Tong', Fengda Zhang", Junkun Yuan', Jun Xiao', Kun Kuang" "Zeejangg University, "Nanyang Technological University {zj hu, tyz j J@ ju.u edu. en, 32zhang3280gmm com, yyanjk0921@our look. com, junxêcs z ju. edu. cn, kunkuangez euuuu..uuuuuuuuuuu.mm edu. .n
Diffusion Model
ASYDM ADM
Diffosion Model
ASYNDM
A sign that AsynDM
rubbit playing baskethall
upple and backpack.
cahe
Three shoep walking together following trail.
big giraffe some leaves
Noloe Level
Synchronous Denoising
Asynchronous Denoising
Figure I: Existing diffusion models generate images through synchronous denoising, where all pix- els are simultaneously denoised step-by-step from noises images, hindering text-to-image align- ment. Asynchronous diffusion models denoise the prompt-related regions more gradually than other regions, thereby receiving clearer inter-pixel context and ultimately achieving improved alignment.
ABSTRACT
Diffusion models have achieved impressive results in generating high-quality im- ages. Yet, they often struggle to faithfully align the generated images with the input prompts. This limitation is associated with synchronous denoising, where all pixels simultaneously evolve from random noise clear images. As result, during generation, the prompt-related regions can only reference the unrelated regions the same noise level, failing to obtain clear context and ultimately im- pairing text-to- -maie alignment. To address this issue, we propose asynchronous diffusion models a novel framework that allocates distinct timesteps differ- ent pixels and reformulates the pixel-wise denoising process. By dynamically modulating the timestep schedules of individual pixels, prompt-related regions are denoised more gradually than unrelated regions, thereby allowing them to leverage clearer inter-pixel context. Consequently, these prompt-related regions achieve better alignment in the final images. Extensive experiments demonstrate that our asynchronous diffusion models can significantly improve text-to-imen to-image aligmment across diverse prompts. The code repository for this work is available at htttps::/githuub.com wiggg///iiiiiininiiiiiiin/
"Corresponding author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Diffusion models have achieved remarkable success across a wide range of domains, such as robotics (Chi et al. 2024 Wolf al. 22025), classification (Li et al. 2023a;
Tong et al. 2025a), im-
age perception and segmentation (Amit et al. 2021 Yuan et al. 2022 2023 text generation (Austin
al. 202T; Nie al., 2025), and visual generation (Yang et al. 2023 Wang et al. 2025). Among these, text-to-image generation has emerged as the most widely recognized application, with the generated images demonstrating impressive diversity and high fidelity (Ho al., 2020; Rombach et al. 2022). Despite their success, even the most advanced diffusion models still struggle with the issue of text-to-image misalignment (Hinz et al. 20200] Ramesh al. 2022: Feng et al., 2023] Chefer
et al., 2023), where the generated images often fail to faithfully match the user-provided prompts,
for example with respect to text, color, or count, as illustrated in Figure
We argue that misalignment in diffusion models is closely associated with the issue of synchronous denoising. That is, under the formulation of a Markov decision process (Ho et al., 2020 Song et al, 2022). all pixels in image simultancously evolve from random noise to clear state, following the same timestep schedule. At each denoising step, pixels interact by leveraging one another as contextual references, ultimately forming coherent and harmonious image.
Beyond this, an image is composed of diverse regions. Some of these regions correspond directly to the objects described in the prompt, while others serve as background. For aligned generation, prompt-related regions typically demand more gradual refinement to accurately capture fine-grained semantics. In contrast, prompt-unrelated regions involve fewer semantic constraints and mainly pro- vide supporting context, allowing them to be denoised into a clear state relatively quickly. However, synchronous denoising treats all pixels equally, overlooking the heterogeneous nature of different regions. Consequently, these prompt-related regions always rely on other regions at the same noise level for contextual references. This raises the concern that synchronuous denoising limits the effec- tive utilization of inter-pixel context, and ultimately hinders text-to-
aligument.
Based on the above motivation, we propose Asynchronous Diffusion Models (AsynDM), a plug- and-play and tuning-free framework that reformulates the denoising process of pre-trained diffusion models. Instead of denoising all pixels simultaneously, the asynchronous diffusion model allows different pixels to be denoised according to varying timestep schedules, as shown in Figure In particular, prompt-unrelated regions can be denoised more quickly, while prompt-related regions denoised more gradually ensure sufficient refinement for capturing prompt semanties. These clearer unrelated regions prevent noisy and ambiguous context from bringing uncertainty to the related regions (e.g., undetermined style, shape, etc.). As a result, the related regions can better focus on the content specified by the prompt, thereby enhancing text-to-image alignment.
Moreover, we introduce a method that dynamically identifies the prompt-related regions and mod- ulates the timestep schedules along the denoising process. Specifically, the cross-attention mod- ules (Vaswani et al., 2017) in diffusion models encapsulate rich information about the shapes and structures of the generated images. At each denoising step, we can extract a mask from the cross- attention modules, which highlights the objects in the prompt. Guided by this mask, the asyn- chronous diffusion model adaptively modulates the timestep schedules of different regions. The highlighted regions (i.e., prompt-related regions) are modulated to be denoised more gradually than other regions (i.e., prompt-unrelated regions), thereby receiving clearer inter-pixel context.
We conduct experiments on four sets of commonly used prompts and compare with advanced base- lines. The results show that AsynDM can effectively improve cext-to-image alignment both quali- tatively and quantitatively. Mcanwhile, AsynDM maintains comparable sampling efficiency to the vanilla diffusion model, it only requires the additional encoding of pixel-wise timesteps.
The main contributions of this paper can be summerized as follows: (1) We highlight that syn- chronous denoising is primary reason for the text-to-image misalignment in existing diffusion models. (2) We propose asynchronous diffusion models that introduces pixel-level timesteps, and adaptively modulate the timestep schedules of different pixels, to address the above issue. (3) Com- prehensive experiments demonstrate that asynchronous diffusion models consistently improve text- to-image alignment across diverse prompts.
BACKGROUND
2

---

Published as a conference paper at ICLR 2026
(a) Asynchronous Diffusion Models for Clearer Inter- Pixel Context
Intermediate Image
Pre-trained Diffusion Model
: Pixel-Level Tinesteps
Attention Block
M, Cross-Atiention Mask
Normalization
Cross- Cross-Allation Aall--llaatiio
X
X;
SelF-Attention
Xi+1
Xy
Concave Scheduler f
Mo x Mir1)
Lincar Schoduler
Aligned Generation Asynchronous Diffusion Models
M
Figure 2: Asynchronous diffusion models improve text-to-in rext-to-image alignment by (a) assigning distinct timesteps different pixels, where faster-denoised regions provide clearer context, serving as better references for slower ones, and (b) using masks extracted from cross-attention to identify prompt- related regions and dynamically modulate pixel-level timestep schedules.
2.1 TEXT-TO-IMAGE DIFFUSION MODELS
Diffusion Model Formulation. Diffusion models have emerged as powerful family of text-to- image generative models. DDPM (Ho al. 2020) formulates the generation process as a Markovian sequence latent states. By denoising step by step, these models progressively transform random noise into coherent image. Based on the DDPM sampler, at each denoising step, the model predicts the last intermediate state x,-1 from current state x; according to:
Po Xx: c) (xx μe tx,,
t, of1
1 1 (x:
Be VI-
with ne (xc, c)
ai -
(2)
Vat
where co denotes the denoising model paramiterized by 6, is the prompt, and dt, 0g and 3r timestep-dependent constants. Subsequent extensions, such as DDIM (Song et al. 2022) and DPM- Solver (Lu al. 2022). further enhance the efficiency and sample quality. These formulations act as the foundation of most modern diffusion- generative models (Rombach et al. 2022).
Attention Module in Diffusion Models.
The attention mechanism /Vaswani et al.
2017) has played
an important role not only in large language models (Zhao et al., 2025] Han et al.
2025 but also in
text-to-image diffusion models (Hertz et al. 2023 Tumanyan et al. (2023). Both bNet-based (Rom-
bach et al. 2022 Podell et al.) 2023) and DiT-based (Peebles & Xie 2023 Esser et al.) 2024) diffu- typical attention block includes a self-attention part and cross-attention part, and can be formally expressed as:
sion models employ attention blocks to enhance expressiveness.
QKT Vdiey VV,
Attention (Q, V) softmax(
where Q Muuuy denotes queries projected from image features, and K RRXxdu, V Buxdaae denote keys and values, projected either from image features (in self-attention) or from prompt embeddings (in cross-attention). Cross-attentios attention allows the models to condition image generation textual prompts, while self-attention further enables the models to capture long-range dependencies across the pixels.
2.2 DIFFUSION MODEL ALIGNMENT
Text-to-in image misalignment has been longstanding challenge across various generative models, including VAËs, GANs and diffusion models (Zhang & Peng 2018 Wang al. 2021 [iao al.
3

---

Published as a conference paper at ICLR 2026
2022). Early diffusion model studies have explored methods for conditioning generation specific factors, such as class labels (Dhariwal & Nichol [0021] Wang et al. 2023b). styles (Sohn et al. 2023)
and layouts (Zheng al. 2023). The incorporation of text encoders has endowed diftusion models with the capability to generate images from textual descriptions Rombach et al. (022). Following this development, recent studies therefore focus on the challenge of text-to-image misalignment in
(Rrmmiaa
diffusion models, which is essential for the reliable deployment.
On the one hand, researchers have sought to achieve better alignment through fine-tuning. Some studies focus on directly fine-tuning the model (Lee et al. 2023 Tong el al., 2025b), among which reinforcement learning-basee methods stand out (Fan et al. 2023 Hu al. 2025a b). Others op- timize different components without altering the main model parameters. For example, some pro- gressively refine the intermediate noisy images during the denoising process (Chefer et al. 2023 Li et al., 2023c, Rassin et al., (023), while others optimize prompts to be more precise and informa- tive Wang et al., 2023a Mañas et al. 2024).
On the other hand, some studies investigate alignment techniques that do not require fine-tuning. For instance, Z-Sampling (LiChen et al., (025) enhances alignment by introducing zigzag diffusion step. SEG (Hong 2024) exploits the energy-based perspective of self-attention
improve image
generation. S-CFG (Shen et al.) 2024) and CFG++ (Chung et al., 2025) improve text-to-image alignment by refining the classifier ffree guidance technique (Ho & Salimans 2022).
3 ASYNCHRONOUS DENOISING FOR CLEARER INTER- PIXELL CONTEXT
In this section, we first introduce the rationale and methodology for allocating distinct timesteps to pixels. We then describe our approach to scheduling the pixel-Ievel timesteps in asynchronous diffusion models. The overview of this section is shown Figure (a).
3.1 PIXEL-LEVEL TIMESTEP ALLOCATION
is reasonable to allocate distinct timesteps to different pixels. During the denoising process of dif- fusion models, image features establish inter-pixel dependencies through the attention mechanism, thus pixels can interact with each other and form a coherent image. Notably, timestep information is embedded into the features in pixel-wise manner external to the attention modules, rather than being directly injected into the attention. In other words, timesteps are involved only in intra-pixel computations, which naturally allows different pixels to be associated with distinct timesteps.
We present the pixel-Ievel timestep formulation of the DDPM sampler, as follows[] We adopt E [0. T' as the new index of the denoising process, since different pixels have distinct timesteps t. This formulation performs denoising from 0 to T, rather than from T to Accordingly, the model predicts the next state Xi+1 from current state x,, where Xi, Xi+1 E B.xxw
po (xi++ X;, c) = -xx++
Ho (xi, L, c), o)),
(4)
(xo )es (xi, C),
Be
with
(x:
(5)
Vote
where t; E Rhxw denotes the timestep states assigned to individual pixels. Specifically, on Se. and ãt, denote element-wise indexing, where each entry of t; selects corresponding scalar value, yielding matrices of the same shape as t, These constant matrices are automatically broadcast along the channel dimension, enabling joint computations with x;. Moreover, the denoising model eg can be seamlessly extended to handle pixel-level timesteps by independently encoding them and incorporating the resulting embeddings into the original computation on a per-pixel basis.
The above formulation enables diffusion models to incorporate pixel-level timesteps. Importantly, the asynchronous diffusion model still preserves the Markov property. the asynchronous setting, t, becomes a tensor with the same height and width as x;, serving as a state within the Markov chain, rather than its original role as the reverse-time index.
'The pixel-level timestep formulation generalize across diverse diffusion samplers. We also provide the formulation of DDIM sampler Appendix A.2
4

---

Published as a conference paper at ICLR 2026
3.2 TIMESTEP SCHEDULING IN ASYNCHRONOUS DIFFUSION MODELS
During the denoising process of diffusion models, the noise level of individual pixels gradually decreases as the timestep progresses from T O. In conventional diffusion models, all pixels share the same timestep scheduler from 7 to O, and commonly used samplers, such as DDPM and DDIM, typically implement this progression linearly. In this subsection, we schedule the timesteps and allow certain regions to evolve more slowly than others. This scheduling enables these regions to accumulate clearer inter-pixel context, thereby achieving more gradual refinement.
We adopt the concave function = f() as the scheduler, according to Proposition
Proposition 1. (See proof in Appendix A.1) Let f(i) [0, - R be a concave function with f(0) = T and f(T) = O. For any in with 0 < in < T and any to with T - in S to S f(o), there exist unique constants o, such that the shifted function fūi a) + b satisfies:
f(io - +b= to:
f(T a) + b=0
(6)
As illustrated in Figure 3 this proposition states that any point located within the shaded area can reach t along the ap- propriately shifted concave function. In the asynchronous dif- fusion model, pixels within the target regions (i.e., the prompt- related regions in text-to-image alignment task) are denoised according to the concave function. By applying only a shift to the concave function, regions selected earlier as targets noised at a slower rate. Other regions, in contrast, are denoised following linear function (or less concave function in some samplers). Therefore, the target regions can be denoised more gradually, thus receive clearer inter-pixel context.
= f())
egg. (= 1==T-1- T
Timestop State
t=T-1
de-
t= - 0)
Denoising Step
T
i
Figure 3: Any point located within the shaded area can reach / = 0
From the perspective of a Markov decision process, in the con- ventional synchronous diffusion models, the state x, transi- tions to the next state X;-1 under the policy distribution pa.
along appropriately shifted
Differently, the state in the asynchronous diffusion model is composed of (x,, t.), which transitions
to the next state (xi+1, t;+1) under the policy distribution (po, f). In our experiments, we simply adopt a quadratic function f(i) T f2 as the scheduling function.
ALIGNED GENERATION VIA ASYNCHRONOUS DIFFUSION MODELS
In this section, we introduce a method that dynamically identifies the prompt-related regions and modulates the timestep schedules of individual pixels along the denoising process.
Prompt-Related Region Extraction. In most rext-to-image diffusion models, cross-attention em- ployed to condition image generation on textual prompts. Even for DiT-based models that rely solely on self-attention, the prompt embeddings are concatenated with image features, thereby enabling implicit cross-attention computations within the self-attention modules (Peebles & Xie 2023).
In cross-attention computation, the term softmax
QK is commonly referred to as cross-attention /heay
maps, denoted by A Rlexxxx where |c] is the number of tokens in prompt c. Previous stud- ies (Tang et al. 2023) Hertz et ( et al. (023) show that cross-attention maps encapsulate rich information about the shapes and structures the generated images. Specifically, the o-th map in A, denoted by A highlights the pixels most influenced by the o-th token. This property allows us to extract a mask that identifies the image regions most relevant to the prompt, as follows:
M = V [1(A>> A
oEO.
where Oe denotes the set of token indices corresponding to the objects described in prompt For each token o, Afnea represents the average value its cross-attention map A 1[:] is the indicator function that produces binary mask based on the given condition, and the operator V indicates an deement-wise logical OR across the resulting masks. This formula ultimately yields a mask that highlights the reempt-related regions.
5

---

Published as a conference paper at ICLR 2026
DM
DM Z-Sampling
SEG
SCFG
CFG+
AsynDM (Ours)
Prompt (Behavior): shark riding bike
Prompt (Count): An opened box four chocolate bananas.
Prompt (Color): A white car and red sheep.
Prompt (Co-occurrence)): a photo of couch and horse
Figure 4: The samples generated by AsynDM and baseline methods across diverse prompts. The images generated by AsynDM show better text-to-image alignment.
Mask-Guided Asynchronous Denoising. At each denoising step we can extract mask M; according to Eq.(7). As illustrated in Figure (b), each mask serves guidance signal for the next denoising step, where the highlighted regions follow the concave scheduler, and the remaining regions follow the linear scheduler. As denoising progresses, the image gradually becomes clearer coarse-to- to-eee manner (Park al.) 2023 Rissanen et al. 2023), and the mask correspondingly evolves to precisely indicate the shapes and positions of the objects. Consequently, the object-related regions are dynamically modulated to denoise more slowly and gradually, thereby receiving clearer inter-pixel context. The clearer context enables these relate- regions better focus the content specified by the prompt, ultimately yielding more faithful and aligned image generation.
EXPERIMENTS
In this section, we first introduce our experimental setting. Next, we demonstrate the effectiveness of AsynDM in improving text-to-imane to-image alignment, providing both qualitative and quantitative results across diverse prompts and comparison with multiple baselines. Finally, we conduct ablation on the mask and the concave scheduler, demonstrating the effectiveness and robustness of AsynDM.
5.1 EXPERIMENTAL SETTING
Diffusion Models. We adopt Stable Diffusion (SD) 2. 2.1-512-base (Rombach et al. 2022), one of the commonly used UNet-based diffusion models, as the foundation model of our experiments. The total timesteps I' is set to 50. We employ the DDIM sampler (Song et al. 2022), and the noise weight 7 is set to 1.0, which determines the extent of randomness at each denoising step. We also conduct advanced diffusion models, including the UNet-based SDXL-base-1A. 1.0 (Podell) et al. 2023) and Dis-based SD3.5-mediie (Esser et al. 2024). The experimental results on these models shown in Appendix [.I
experiments
Prompts. We adopt four commonly used prompt sets in our experiments. (1) Animal activity (Blakk
al. 2023). This prompt has the form "a(n) [animal] [activity]", where the activities come from

---

Published as a conference paper at ICLR 2026
Table 1: Text-to-imaae alignment performance of AsynDM compared with baseline methods across
diverse prompts.
Prompt Set
Method
BERTScoref
CLIPScoref
ImageRewardf
QwenScoref
DM
0.6353
0,3685
0.7543
4.9445
DM
0.6381 (+0.0028)
0.3715 (-0.0030)
0.8544 (-0.1001)
0,0655 (+0.1250)
7-Sampling
0.6353 (+0.0000)
0.3708 (-0.0023)
0.8283 (+0.0740)
5.0242 (+0.0797))
Animal Activity
SEG
0.6309 (-0.0044)
0,3605 (-0.0080)
0.6493 (-0.1050)
4,7632 (-0.1813)
S-CFG
0.6383 (+0.0030)
0.3716 (+0.0031)
0.8653 (+0.1110)
5.0421 (40.0976)
CPG++
0.6249 (-0.0104)
0.3565 (-0.0120)
0.3284 (-0-4259)
4,4484 (-0.4961)
AsynDM
0.6414 (+0.0061)
0.3750 (+0.0065)
0.9219 (+0.1676)
5.5218 (40.5773)
DM
0.6968
0,3689
0,3943
4,7406
DM
0.6970 (+9.0002)
0.3670 (-0.0011)
0.4152 (+0.0209)
4,8179 (+0.0773)
Z-Sampling
0.6979 (+0.0011)
0.3676 (-0.0017)
0.4505 (+0.0562)
4,7656 (+0.0250)
Drawbench
SEG
0.6925 (-0.0043)
0.3527 (-0.0132)
0.2478 (-0. 1465)
4.6695 (-0.071
S-CFG
0.6972 (+0.0004)
0.3693 (-0.0034)
0,4398 (+0.0455)
4,8750 (40.1344)
CPG++
0.6938 (-0.0030)
0.3539 (-0.0120)
0.1644 (-0.2299)
4,6210 (-0.1196)
AsynDM
0.7007 (+0.0039)
0.3701 (+0.0042)
0.4560 (+0.0617)
4.9804 (+0.2398)
DM
0,7030
0.3620
0.1541
4.9390
DMconcave
0.7039 (+0.0009)
0.3637 (-0.0017)
0.1979 (+0.0438)
4,9976 (+0.0586)
Z-Sampling
0.7046 (+0.0016)
0.3626 (-0.0006)
0.1757 (+0.0216)
4,9179 (.0.021
GenEvul
SEG
0.7005 (-0.0025)
0,3493 (-0.0127)
0.0689 (-0.0852)
4,9125 (-0.0265)
S-CFG
0.7031 (+0.0001)
0.3630 (-0.0010)
0.1819 (+0.0278)
4.8968 (-0.0422)
CFG++
0.6992 (-0.0038)
0.3482 (-0.0138)
-0.1344 (-0.2885)
4.5835 (.0.3555)
AsynDM
0.7081 (+0.0051)
0.3683 (=0.0063)
0.2895 (+0.1354)
5.3390 (+0.4000)
DM
0.6995
0,3388
0,2696
5,8507
DMoecsse
0.7004 (+0.0009)
0.3395 (+0.0007)
0.2917 (+0.0221)
5,9632 (40.1125)
Z-Sampling
0.6999 (+0.0004)
0.3377 (-0.0011)
0.2946 (+0.0250)
5,8289 (-0.0218)
MSCOCO
SEG
0.6952 (40.0043)
0.3295 (-0.0093)
0.1667 (-0.1029)
5.8320 (.0.0187)
S-CFG
0.6995 (+0.0000)
0.3409 (-0.0021)
0.3316 (+0.0620)
5,9328 (+0.0821)
CFG++
0.6975 (-0.0020)
0.3348 (-0.0040)
0.1471 (-0. 1225)
5.6921 (-0.1586)
AsynDM
0.7055 (+0.0060)
0.3420 (=0.0032)
0.3339 (+0.0643)
6.2601 (+0.4094)
humans, such "riding bike". (2) Drawbench (Saharia et al. 2022). This prompt consists of 11 categories with approximately 200 prompts, including aspects such as color and count. (3) GenEval (Ghosh et al.] 2023). This prompt set incorporates 553 prompts, including aspects such as co-occurrence, color and count. (4) MSCOCO (Lin et al. 2014.. This prompt set is derived from the captions of the MSCOCO 2014 validation set and consists of descriptions of real-world images. For each set, we randomly select 40 prompts for our experiments.
Metrics. In our experiments, we employ four metrics to evaluate text-to-image alignment. (1) BERTScore (Zhang al. 2020). This metric leverages multimodal large language model to gen- erate a description for the image, and then employs BERT-based recall to quantify the semantic sim- ilarity between the prompt and the generated description. In our implementation.
use Qwen2.5- VL-7B-Instruct (Wang et al. (024) to generate descriptions and DeBERTa xlarge model (He et al.] 2021) to compute similarity. (2) CLIPScore. This metric measures the similarity between the text embeddings and image embeddings encoded by CLIP model (Radford et al. 2021). We use ViT- H-14 CLIP model in implementation. (3) ImageReward (Xu et al. | 2023). This metric employs a pre-trained model to estimate human preferences, in which alignment serves as a key factor. (4) QwenScore. We employ Swen2.5-VLL-7B-nnsteee
(Wang al., 2024) to score text-to-image align- ment directly, ranging from 0 to 9. The prompts fed to Qwen are provided in Appendix B.4
Baselines. We sample the diffusion model using both the standard scheduler and the concave sched- uler, denoted as DM and DMconeave- respectively. In addition, we compare AsynDM with the most
advanced methods, including Z-Sampling (LiChen et al. 2025) SEG (Hong 2024 S-CFG (henn
et al. 2024) and CFG++ (Chung et al. 2025).
5.2
QUALITATIVE EVALUATION
We first provide the qualitative results of AsynDM in comparison with multiple baselines, as shown in Figure 4 We select several representative prompts that encompass object behavior, count, color,
7

---

Published as a conference paper at ICLR 2026
Human Prefercnce Rate
81.97%
72.60%
93.83%
73.32%
0.6
0.4
17.55%
9.86%
1.92%
6.25%
7.69%
18.99%
12.74%
5.29%
Animal Activity
Drawbench
GenEval
MSCOCO
DM
DMconcave
AsynDM
Figure 5: Human preference rates for text-to-image alignment of the images generated by DM, DMcnevve and AsynDM.
Table 2: Text-to-image alignment performance of AsynDM when employing different concave schedulers and using fixed masks, across prompts from animal activity set.
Method
BERTScoret
CLIPScoref
ImageRewardf
QwenScoref
Scheduler
DM
0.6353
0,3685
0.7543
4.9445
DMonuuue
(+0.0028)
0.3715 (-0.0030)
0.8544 (+0.1001)
5.0695 (+0.1250)
Quadratic
AsynDM
0.6414 (+0.0061)
0.3750 (+0.0065)
0.9219 (+0.1676)
5.5218 (+0.5773)
*ftred mask
0.6405 (+0.0052)
0.3722 (-0.0037)
0.8642 (+0.1099)
5.2593 (40.3148)
DMconewee
0.6338 (-0.0015)
0,3667 (-0.0018)
0.7043 (-0.0500)
4,7406 (-0.2039)
Piecewise Linear
AsynDM
0.6401 (+0.0048)
0.3724 (+0.0039)
0.8472 (+0.0929)
5,2335 (+0.2890)
ffxed mask
0.6383 (+0.0030)
0.3705 (-0.0020)
0.7504 (-0.0039)
5,0812 (40.1367)
DMoneese
0.6352 (-0.0001)
0.3689 (-0.0004)
0.7981 (+0.0438)
4,9289 (-0.0156)
Exponential
AsynDM
0.6408 (+0.0055)
0.3715 (+0.0030)
0.8686 (+0.1143)
5.2367 (+0.2922)
+fixed mask
0.6386 (+0.0033)
0.37 14 (-0.0029)
0.8374 (+0.0831)
5.2023 (+0.2578)
and co-occurrence. The vanilla diffusion model (i.e., DM and DM fails to generate images that well aligned with the prompts. In contrast, AsynDM effectively generates well-aligned images with the same random seeds. Additional qualitative examples, together with those from SDXL and SD 3.5, can be found in Appendix
5.3 QUANTITATIVE EVALUATION
We also quantitatively demonstrate the text-to-image alignment performance of AsynDM compared with baseline methods. As shown in Table [ we sample 1,280 images for each of the four prompt sets, using the same random seeds across different methods. The generated images are then evalu- ated with four metrics. The results demonstrate that AsynDM consistently achieves better alignment across all prompt sets. Meanwhile, sampling 1,280 images takes 78 minutes using the vanilla dif- fusion model, compared to 86 minutes using AsynDM, which indicates that AsynDM achieves im- provements without significantly sacrificing efficiency. In addition, we conduct human evaluation. We invite 52 participants choose the image they consider best aligned with the prompt from each group of three candidates, corresponding to DM, DMonnaave and AsynDM. As shown in Figure the results further demonstrate that AsynDM improves text-to-image alignment.
We also evaluate the image quality of AsynDM using FID-30K (L). FID-30K refers to the Frechet Inception Distance calculated using 30,000 images from the MSCOCO 2024 validation set as the reference dataset (Pavlov et al. [2023 Lin et al. 2014). We merge all four prompt sets and generate 16,000 images with each of DM. DM Coocavve and AsynDM. The resulting FID-30K scores 48.65 48.63 for DM, 49.29 for DM and 49.3 for AsynDM. These results indicate that our method can largely preserve the image quality of the pretrained diffusion model.
5.4 ABLATION STUDY
Ablation on Mask. In this ablation study, we replace the dynamically updated mask with a fixed mask. This fixed mask is extracted from the average cross-attention map of DM during its denoising process, following 77. Due to the use of the same random seed, the mask derived from DM can roughly highlight the prompt-related regions in the image generated by AsynDM. The results
8

---

Published as a conference paper at ICLR 2026
DM
Different Seed
AsynDM
Different Seed
AsynDM
a Asynchronous Diffusion Models for Reducing Image Distortion
Base Image
FLUXX
AsynFL.UX
Base Image
FLUX
AsynFLUX
S S
DFFLUSION WODEL
DIFFUSION MODEL
Replace the text on the sign with 'Diffuston Model'
Replace the baskethall with a yellow toy train
The shark drives a car instead of bicycle now (b) Asynchronous Diffusion Models for Enhancing Editing Performance
Turn red car sheep
Figure 6: We further employ AsynDM to reduce image distortion and enhance editing performance.
shown in the Table 2 Despite the fixed mask being imperfect, AsynDM still improves to-too--mmae alignment compared with the base model, demonstrating its robustness to inaccurate masks.
Ablation on Concave Scheduler. addition to the quadratic scheduler, we also employ the piece- wise linear scheduler and the exponential scheduler to AsynDM, as follows:
3
f(i) min (T
-1, -T
2 i),
(Piecewise Linear Scheduler)
2
2
f(()
(e -0--et) et)).
(Exponential Scheduler)
As shown in Table 2. AsynDM consistently improves image alignment across different schedulers. This is because, across all the variants, these concave schedulers enable the prompt-related regions to receive clearer inter- inter-pixel context. These results further demonstrate the effectiveness and robustness of AsynDM. The image samples of these two ablation studies are provided in Appendix
6 FURTHER EXPLORATION AND DISCUSSION
Asynchronous Diffusion Models for Reducing Image Distortion. Diffusion-generated images often suffer from distortions, such as abnormal limb shapes. shown in Figure inpainting the distorted regions under different random seeds yields limited improvements. In contrast, applying AsynDM with a mask over the distorted regions, while using the same seed, generates improved images. This suggests that AsynDM has the potential to mitigate image distortions.
Asynchronous Diffusion Models for Enhancing Editing Performance. FLUX.1 Kontext a DiT- based diffusion model that unifies image generation and editing (Labs et al. 2025). However, as shown in F (b), even this advanced model can produce edits that mismatch the user prompts. By manually annotating the regions be edited and applying the concave scheduler during the

---

Published as a conference paper at ICLR 2026
editing process, the resulting images align more closely with user expectations. This observation suggests that AsynDM has the potential to further enhance the performance of image editing models.
Limitations and Future Work. (1) In this work, employ fixed concave function to guide the transition of timestep states, A promising direction for future research is to replace this fixed function with learnalle model that can adaptively predict the next timestep state for each pixel (e.g., Ye et al. (2025) Li et al. (203bb), potentially leading to more flexible and accurate transitions. (2) We only distinguish between prompt-related and unrelated regions. A natural extension would be to capture more complex object relationships by sorting the objects or constructing a directed acyclic graph (Han al. 2024 Kong et al. 2025). Assigning different objects with varying concave schedulers may further lead to improved performance. (3) When timestep states across pixels differ extremely, the faster denoised regions may be affected by noisy regions, causing the final image to retain considerable amount of noise (See Appendix D.2 for an example). We attribute this limitation to the training-free nature of AsynDM, which makes less robust to large disparities in noise levels. Future work could address this issue through fine-tuning or pre-training.
7 CONCLUSION
In this work, we propose the asynchronous denoising diffusion models to improve text-to-image alignment. The AsynDM allocates distinct timesteps individual pixels and schedules them using a concave function. Guided by the masks that highlight the prompt-related regions, these regions can be denoised more slowly than unrelated ones, allowing them to receive clearer inter-pixel context. The clearer context can help the related regions better capture the content specified by the prompts, thereby generating more aligned images. Our empirical results demonstrate the effectiveness and robustness of the proposed asynchronous diffusion models.
ACKNOWLEDGEMENTS
This work was supported in part by the National Natural Science Foundation of China (62376243), the National Key Research and Development Program of China (2024YFE0203700, and "Pioneer" and "Leading Goose" R&D Program of Zhejiang (2025C02037), All opinions in this paper are those of the authors and donot necessarily reflect the views of the funding agencies.
ETHICS STATEMENT
This paper aims to advance the broader field of text-to-image alignment in diffusion models. While our method focuses on improving controllability and semantic faithfulness in generative models, may have societal implications similar to those associated with image synthesis technologies. We do not identify any concerns unique to our approach that require special emphasis. For a more extensive discussion of the ethical considerations and broader impacts surrounding diffusion models and text-to-image generation, we refer interested readers to Po et al. (0024).
REPRODUCIBILITY STATEMENT
We have taken several measures to ensure the reproducibility of our work. The detailed experimental setting is provided in Section 5.1 of the main paper, and the Appendix B includes comprehensive implementation details, such as hyperparameters. To further ensure reproducibility, we provide pseudo-code that outlines the proposed method step by step in Appendix
REFERENCES
Tomer Amit, Tal Shaharbany, Eliya Nachmani, and Lior Wolf. Segdiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390, 2021.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing 2021.
systems,
10

---

Published as a conference paper at ICLR 2026
Kevin Black, Michael Janner, Yilun Du, llya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 13301, 2023.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Mas- actrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 22560 22500, 2023.
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-eecite: Attention-based semantic guidance for text-to-image diffusion models. ACM transactions Graphics (TOG), 42(4):1-10, 2023.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024. URL
httts:://rrx: www./og/////ooooooo//t...t.o.as/
abs
Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold-consttaineee classifier free guidance for diffusion models. In The Thirteenth Interna- tional Conference Learning Representations, 2025. URL https: pestps://openneview..nr
Eorum?id=E7 7
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 -77944 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resouution image synthesis. Forty-first international conference on machine learning, 2024.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Picter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lec. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 06.79858-79885 2023.
Weixi Feng, Xuehai He. Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https:/ /tpp///peeeeee.ee/o//o/ee / PUIqiT4rzq7
net forum?id=
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52152, 2023.
Kairong Han, Kun Kuang, Ziyu Zhao, Junjian Ye, and Fei Wu. Causal agent based on large language model, 2024. URL /ttttps:/aaxxx..ot abt//408..222//.../// 06849
Kairong Han, Wenshuo Zhao, Ziyu Zhao, JunJian Ye, Lujia Pan, and Kun Kuang. Cat: Causal attention tuning for injecting fine-grained causal knowledge into large language models, 2025. URL
org. abs 01535
Pengcheng He, Xiaodong Liu. Jianfeng Gao, and Weizhu Chen. Deberta: ercoding-enhanee bert with disentangled attention. In International Conference on Learning Representations, 2021.
URL
net./ wtt.rrrmmmtoooottttttttt.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-promp image editing with cross-attention control. In The Eleventh International Con- ference on Learning Representations, 2023. URL. https: /ttp:///eeeeeeeeeeeemmoe /
formf?
id= xzkzeyb
Tobias Hinz, Stefan Heinrich, and Stefan Wermter. Semantic object accuracy for generative text-to- image synthesis. IEEE transactions on pattern analysis and machine intelligence, 44(3):1552- 1565, 2020.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho and Tim Salimans, Classifier-free diffusion guidance, 2022. URL https:
orq/aba aaaaab///e 2207.1252
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-666 6851, 2020.
Susung Hong. Smoothed energy guidance: Guiding diffusion models with reduced energy curvature of attention. Advances in Neural Information Processing Systems, 77766743-66777 2024.
Zijing Hu, Fengda Zhang, Long Chen, Kun Kuang, Jiahui Li, Kaifeng Gao, Jun Xiao, Xin Wang, and Wenwu Zhu. Towards better alignment: Training diffusion models with reinforcement learning against sparse rewards. In Proceedings of the Computer Vision and Pattern Recognition Confer- ence, pp.
2025a.
Zijing Hu, Fengda Zhang, and Kun Kuang. D-fusion: Direct preference optimization for aligning diffusion models with visually consistent samples. In Forty-second International Conference on Machine Learning, 2025b. /ttphttts:///peereeee
ew. net forum? d=WV EwF DGH
Lingjing Kong, Guangyi Chen, Biwei Huang, Eric P. Xing, Yuejie Chi, and Kun Zhang. Learn ing discrete concepts in latent hierarchical models, 2025. BRLhhtpps:/ /aattps://axiv...e www. abs/
2406. 00519
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux. kontext: Flow matching for in-context image generation and editing latent space, 2025. URL URL https::tt/sssett
orq/ abs 2200/5566.00 15742
Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback. arXiv preprint 2rXiv:2302.12199 2023.
Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your dif- fusion model is secretly zero-shot classifier. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2206-2217,200 2023a.
Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Ji. Autodiffusion: Training-free optimization of time steps and architec- tures for automated diffusion model acceleration. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 7105-7114, 2023b.
Yumeng Li, Margret Keuper, Dan Zhang, and Anna Khoreva. Divide & bind your attention for improved generative semantic nursing. arXiv preprint urXiv:2307. 10864, 2023c.
Wentong Liao, Kai Hu, Michael Ying Yang, and Bodo Rosenhahn. Text to image generation with semantic-spatial aware gan. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 18187-18198 2022.
Bai LiChen, Shitong Shao, zikai zhou, Zipeng Qi, zhiqiang xu, Haoyi Xiong, and Zeke Xie. Zigzag diffusion sampling: Diffusion models can self-improve via self-reflection. In The Thirteenth International Conference on Learning Representations, 2025. WtRhhttps::// petttps://openeee /otpnreviee......w...
net/Eorum? www.mmiiid.dddowveeee.e.e...e.ee
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li. and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775-57877 2022.
Oscar Mañas, Pietro Astolfi, Melissa Hall, Candace Ross, Jack Urbanek, Adina Williams, Aish- warya Agrawal, Adriana Romero- -Soiino,, and Michal Drozdzal. Improving text-to-image con- sistency via automatic prompt optimization. arXiv preprint arXiv:2403. /7804, 2024.
12

---

Published as a conference paper at ICLR 2026
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025. URL https://
arxiv.org/ abs 2502 09992
Yong- Hhyng Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Understanding the latent space of diffusion models through the lens of riemannian geometry. Advances in Neural 2023.
Information Processing Systems, htt.t//w...ccoooooom/o////////ee ben.haarggebbccaaak......... September
I. Pavlov, A. Ivanov, and S. Stafievskiy. Text-to-Image Benchmark: A benchmark for genera- tive models. 2023. Version 0.1.0.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4205, 2023.
Ryan Po. Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T Barron, Amit Bermano, Eric Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, al. State of the art diffusion models for visual computing. In Computer graphics forum, volume 43, pp. e15063. Wiley Online Library, 2024.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. 2023. URL http::tt:://ttttt /ttppp:://rxiv...o// org/ abs/ 2307.
Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022. URL https:://arxyy> org/ org/abs//////aaooo abs/ 2204. 06125
Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. Advances in Neural Information Processing Systems, 36:3536-3559, 2023.
Severi Rissanen, Markus Heinonen, and Amo Solin. Generative modelling with inverse heat dis- sipation. In The Eleventh International Conference on Learning Representations, 2023. URL
https: //ppnr/vppevviii iew. net Forum??id=4P.JUTTTT
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural informa- tion processing systems, 35:36479-36444 2022.
Dazhong Shen, Guanglu Song, Zeyue Xue, YunYun Wang. and Yu Liu. Rethinking the spatial inconsistency in classifier-free diffusion guidance. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9370-9317 2024.
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image generation in any style. arXiv preprint arXiv:2306.0000 306.0000000,,,. 2023.
Jamming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, 2022. URL https:://axxi http://wwxwvv.......... org/ abs////b///// 2010.02502
Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stene- torp, Jimmy Lin, and Ferhan Türe. What the daam: Interpreting stable diffusion using cross atten- tion. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5644 -6599, 2023.
13

---

Published as a conference paper at ICLR 2026
Yunze Tong, Fengda Zhang, Zihao Tang, Kaifeng Gao, Kai Huang, Pengfei Lyu, Jun Xiao, and Kun Kuang. Latent ssore-based reweighting for robust classification on imbalanced tabular data. In Proceedings of the 42nd International Conference Machine Learning, 2025a.
Yunze Tong, Fengda Zhang, Didi Zhu, Jun Xiao, and Kun Kuang. Decoding correlation- -nnuuced misalignment in the stable diffusion workflow for text-to-ine text-to-image generation. In Proceedings of the IEEE/CVF international conference on computer vision, 2025b.
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. text-driven image-to -ome-agg translation. In 2023 IEEE/CVF Conference Computer Vision and Patlern Recognition (CVPR), pp. 1921-1930, 2023. doi: 10. 10110/9PR522299... 202.32209..2.001911
diffusion features for
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- lion processing systems, 30, 2017.
Chen Wang, Hao- Yang Peng, Ying-Tian Liu, Jiatao Gu, and Shi-Min Hu. Diffusion models for 3d generation: survey. Computational Visual Media, 11(1):1-28, 2025. doi: 10.26599/CVM. 2025.9450452
Hao Wang, Guosheng Lin, Steven CH Hoi, and Chunyan Miao. Cycle-consistent inverse gan for text-to-image synthesis. In Proceedings of the 29th ACM international conference on multimedia, pp. 630-638, 2021.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191 2024.
Yunlong Wang. Shuyuan Shen, and Brian Y Lim. Reprompt: Automatic prompt editing to refine ai- generative art towards precise expressions. In Proceedings of the 2023 CHII conference on human factors in computing systems. pp. 1-29, 2023a.
Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou, al. Patch diffusion: Faster and more data-efficient training of diffusion models. Advances in neural information processing systems, 36:72137-72154 2023b.
Rosa Wolf, Yitian Shi, Sheng Liu, and Rania Rayyes. Diffusion models for robotic manipulation: A survey, 2025. UtL //tpvvvaavvvvvv/va org/ abs/2504. 08438
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising difffusion GANs. In International Conference on Learning Representations, 2022. URL
pettps://openreviiee net wwt.://.eee/eeeeeeeeeeeee www.foorrdiooooeooo..o.... JpMMp qoCo
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li. Ming Ding. Jie Tang, and Yuxiao
Dong. Imagereward: Learning and evaluating human preferences for eoxttto-image generation.
Advances in Neural Information Processing Systems, 36: -5903-155535 2023.
Ruihan Yang, Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video
generation. Entropy, 25(10):1469, 2023.
Zilyu Ye, Zhiyang Chen, Tiancheng Li, Zemin Huang, Weijian Luo, and Guo-Jun Qi. Schedule on the fly: Diffusion time prediction for faster and better image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 23412-23422, June 2025.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pp. 6613-6612 2024.
Junkun Yuan, Xu Ma, Defang Chen, Kun Kuang, Fei Wu, and Lanfen Lin. Label-efficient domain generalization via collaborative exploration and generalization. In Proceedings of the 30th ACM international conference multimedia, pp. 2361-2370, 2022.
14

---

Published as a conference paper at ICLR 2026
Junkun Yuan, Xinyu Zhang, Hao Zhou, Jian Wang, Zhongwei Qiu, Zhiyin Shao, Shaofeng Zhang, Sifan Long, Kun Kuang, Kun Yao, et al. Hap: Structure-awar masked image modeling for human-centrie perception. Advances in Neural Information Processing Systems, 36:50597- 50616, 2023.
Chenrui Zhang and Yuxin Peng. Stacking vae and gan for context-aware text-to-image generation. In 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), pp. 1-5. IEEE, 2018.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi Bertscore: Evalu- ating text generation with bert, 2020. URL https::/arxiv..nrt org. http:/bb//19044 09675
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian- Yun Nie, and Ji-Rong Wen. A survey of large language models, 2025. URL https: // wttaxxxivv.ooo org.bbs.
23C3.18223
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 22490-22499, 2023.
15

---

Published as a conference paper at ICLR 2026
The Appendix is organized as follows:
Appendix
A
provides the proof of the proposition in the main text and the formulation of the
DDIM sampler.
Appendix
B
provides
details implementation.
Appendix provides the pseudo-code of employing AsynDM to generate images.
Appendix presents more experimental results.
Appendix presents more image samples generated by AsynDM.
Appendix describes the role of the large language models (LLMs) in preparing this paper.
A
THEORETICAL DERIVATIONS
A.1 PROOF OF PROPOSITION
From the second equation in 6 we obtain b = = a). Substituting into the first equation
yields the single-variable condition f(i a) - f[ a) = to- Define:
g(a) = f a) [[T - a),
E [0, in].
(8)
The domain [0, dol ensures that both ip a and T a lie in [0,
continuous, hence continuous on [0) Moreover, concavity implies that the slope of f is nonincreasing, which in turn gives:
Since f is concave on [0, T) then
(a)a) fT a) f a) s 0,
whenever f is differentiable. Therefore, g is nonincreasing on [0, io]. and strictly decreasing unless f is linear.
At the endpoints, have: g(0) f(io) f(T)= f(io), g(io) f(0) f(T 10) =
(10)
Therefore, the range of g exactly the interval [T J(T - [(io)]
Moreover, since f concave on [0, 7). then:
f(T- - in) f( 10 T i0
1g
T i0
0+
1)
2
f(0) +
f(T) 10.
(11)
T
T
T
Hence - - f(T i0) S ig-
According to the intermediate value theorem, for any - in f(i)).. there exists some E [0, iol. such that g(a) = ty. Monotonicity of g guarantees that this solution is unique. Finally, since is uniquely determined, then b = fTT a) is also uniquely determined.
Therefore, the constants a, b exist and are unique.
A.2 ASYNCHRONOUS DENOISING WITH DDIM SAMPLER
The vanilla DDIM sampler predicts next intermediate state x according
X Va-ii Xy + / a11 - of cx or.
(12)
with &o
(xi VI au co cx(x,tt,
(13)
Va
where 6, ~ NO, The pixel-level timestep formulation of the DDIM sampler given as follow:
Xi+1 Viituu 80 + / at... - of ex c) orei,
(14)
1
with 8o
(x; v1 - Ot (x c)),
(15)
/ v
16

---

Published as a conference paper at ICLR 2026
Image Features K1 R (how)xdaay
Prompt Fimbeediigss
Prompt Emboddings K IR lejxdaey
Q E B (oxw)xhey
K2
(exday
Image Fentures
Image Feeuuures
Cross Attention Maps A & (hxw)xld
QE (hoow
Cruss Attention Maps
IR. (txw)xlel
Prompl Embeeklings
Qx R 4xdwyy
(a) Cross-Attention Module
(b) Joint Self-Attention Module
Figure Extracting cross-attention masks from attention modules.
B IMPLEMENTATION DETAILS
B. IMPLEMENTATION DETAILS OF ASYNDM
Mask Extraction. Section we have described how to extract prompt-related regions from cross-attention maps. However, model typically contains multiple cross-attention layers, each pro- ducing its own set of attention maps. For DiT-based diffusion models, we average the cross-attention maps across all layers and then extract the mask following the procedure outlined in Section 4 In contrast, UNet-based diffusion models comprise layers with varying spatial resolutions. Let h x w represent the image resolution Xt, and hy x wy represent the resolution at layer of the UNet. Inspired by prior work (Hertz et al. 2023 Cao et al., 2023), we only use the cross-attention maps from layers at resolution h x wi T
The maps from these layers are averaged obtain the mask, and subsequently upsampled to the resolution h x
shown in Appendix D.2. when timestep states across pixels differ extremely, the prompt-unrelated regions in the final image might retain a considerable amount of noise. Therefore, constraining the maximum disparity of timestep states across pixels fundamental to ensuring that any concave function be reliably applied for denoising. To achieve this, adopt a straightforward yet effective strategy by weighting the concave function f with the standard denoising function, (e.g., the linear function). Consequently, the concave function employed for state transitions becomes I' w (1 w) where w E (0, 1). The function f' not only retains the concavity, but also mitigates its maximum disparity with respect to the standard function.
Scheduler Reweighting.
B.2 DETAILS HUMAN EVALUATION
The human study was conducted with 52 participants from eight universities, including 23 females and 29 males, whose academic backgrounds ranged from undergraduates to Ph.D. students. Each participant was asked to complete form. At the beginning of the form, we provided the following instruction: "Image-prome
alignment refers how well an image matches the given textual de- scription. For each group of three images, please select the you believe best matches the given textual description." " The form contained 64 groups of images in total, corresponding to four prompt sets, each comprising 16 groups. These 64 groups were randomly selected from the images sampled during the evaluation of results reported in Table Each group consisted of three images generated by DM, DMconcave and AsynDM under the same random seed, accompanied by the corresponding text prompt used for generation. All participants received the same set of images, but the presen- tation order was randomized, ensuring that participants were unaware which method each image originated from. The entire form was presented in English.
B.3 EXTRACTING
MASKS FROM DIT-BASED MODELS
As shown Figure 7 (a), the cross-attention modules, we first obtain the cross-attention maps directly via A softmax QK" and subsequently derive the corresponding masks using Eq.07
17

---

Published as a conference paper at ICLR 2026
However, DiT-based diffusion models typically do not include dedicated cross-attention modules. Instead, they rely on implicit cross- tross-attention computation within the self-attention modules to enable the image to be guided by the prompt. As illustrated in Figure 7 (b), the queries Q and keys K are formed by concatenating the image features with the prompt embeddings. During the attention operation, the resulting attention maps Ajoini has size of (h x w (cl) (h ww+ w+/cc]) By extracting the submatrix corresponding to the interactions between the image-feature queries and the prompt- embedding keys, we can obtain the cross-attention maps (i.e., A Ajoin|: (/ w), (/ w) :)). The cross-attention masks are then computed using Eq.
B.4 PROMPT FOR QWEN
We employ Qwen2.5-VLL-7B. Insttrucc (Wang et al. 2024) to score text-to-image alignment with the following prompt: "You are given image and a description. Please evaluate how well the image matches the description on a scale from to 9, where means completely unrelated and means perfectly aligned. Return only the [prompt used to generate the imagel"
single integer without explanation. \n Description:
B.5 EXPERIMENTAL RESOURCES
The experiments were conducted on 24GB NVIDIA 3090 GPUs. It tooks approximately 78 minutes for the vanilla diffusion model SSD2.1-512-bar generate 1,280 images, and approximately 86 minutes for the asynchronous diffusion model.
B.6 HYPERPARAMETERS
The full hyperparameter list of our experiments is presented in Table 3
Table 3: Hyperparameters of our experiments.
Patameter
Value
Denoising steps T Noise weight
1.0
Classifier-free guidance
True
Sampling
Guidance scale
5.0
Batch size
Batch count
160
Inversion guidance "22
0.0
Z-Sampling
Zigzag steps Number of rounds Tmax
49
SEG guidance "seg Blurred weight a
3.0
SEG
1.0
CFG++
CFG++ guidance
0.4
C PSEUDO-CODE
The pseudo-code of employing the asynchronous diffusion model to generate text-aligned images is shown in Algorithm
2 "The cross attenntttenion maps A has size of (/1 w) x /c). The dimension |e| x h x w mentioned in the main text corresponds to its transposed and reshaped form, which is presented to facilitate clearer understanding for the readers.
18

---

Published as a conference paper at ICLR 2026
Algorithm 1: Pseudo-code of employing the asynchronous diffusion model to generate text- aligned images.
Input Total denoising timesteps 7, number of samples N, prompt list C, pre-trained diffusion model 60, linear/standard scheduler g, concave scheduler f.
D Pample = =[];
for n 4- to do
C C
// Initialize x,, t; and M
Randomly choose Xy from (0,
In - lensor(shape(xo). fill 7):
M tensor(shape(xx:
fill 1)
for i f 0 to T do
// Transition
t;
i+1 Calculate the next state of t, using 9:
toon Calculate the next state of t, using f;
6,11 + M 141 + (1 (1-M) ) x 1+1
// Transition
X;
< c0(xi, 1, c), and extract the cross-attention map A
Calculate x according to the chosen sampler (e.g., Eq. for DDPM);
// Update M
Update M using Eq.
end
D sample--app Samppe-append((x) append(xx))
end
Output: saampl
D MORE EXPERIMENTAL RESULTS
D.1 EXPERIMENTS ON SDXL AND SD3.5
We also quantitatively demonstrate the text-to-image alignment performance of AsynDM compared with baseline methods on SDXL and SD 3.5, as shown in Table 4 and Table 5 respectively. For experiments conducted on SD 3.5, we have included comparisons with Z-Sampling CFG++ This is because Z-Sampling relies on DDIM inversion, and CFG++ makes modifications to DDIM. However, SD 3.5 is flow model that is not directly compatible with the DDIM sampler. The experimental results demonstrate that AsynDM consistently achieves better alignment across all prompt sets. The image samples for these experiments are shown Figure 12 and Figure 13
Table 4: Text-to-image alignment performance of AsynDM compared with baseline methods on animal activity prompt set. The base model is 3DXL-base-1 (Podell et al. 2023).
Prompt Set
Method
BERTScoref
CLIPScoret
ImageReward
QwenScore?
DM DMconnase
0.6671
0.3976
1.6552
6,6562
0.6695 (+0.0024)
0.3993 (+0.0017)
1.6768 (-0.0216)
6.8421 (+0.1859)
7-Sampling
0.6674 (+0.0003)
0.4022 (+0.0046)
1.6677 (-0.0125)
6.7320 (+0.0758)
Animal Activity
SEG
0.6673 (+0.0002)
0.3963 (-0.0013)
1.6417 (-0.0135)
6.8085 (40.1523)
S-CFG
0.6670 (-000001)
0.3981 (+0.0005)
1.6481 (-000011)
6.6367 (-0.0195)
CFG++
0.6581 (-0.0090)
0.3879 (-0.0097)
1.3748 (-0.2804)
6.4078 (-0.2484)
AsynDM
0.6829 (+0.0158)
0.4026 (+0.0050)
1.6893 (+0.0441)
7.2781
(+0.6219)
D.2 ABLATION ON MAXIMUM TIMESTEP DIFFERENCE
Given an extreme concave scheduler f(i) min 27 - 2i) and standard linear scheduler g(i)
-, i, the maximum timestep difference between pixels within the same denoising step can reach T.
By interpolating the two schedulers f' + w) g. obtain concave scheduler whose
19

---

Published as a conference paper at ICLR 2026
Table 5: Text-to-image alignment performance of AsynDM compared with baseline methods animal activity prompt set. The base model is
(Esser et al. 2024).
Prompt
Method
BERTScoret
CLIPScoret
ImageRewardf
QwenScore?
DM
0.6590
0.3906
1.5091
6,8812
DMcomavee
0.6603 (+0.0013)
0.3928 (+0.0022)
1.6385 (-0.1294)
7.0656 (40.1844)
Animal Activity
SEG
0.6570 (-0.0020)
0.3740 (-0.0166)
1.4022 (-0.1669)
7.1250 (+0.2438)
S-CFG
0.6629 (+0.0039)
0.3908 (+0.0002)
1.6227 (-0.1136)
7.0125 (+0.1313)
AsynDM
0.6663 (+0.0073)
0.3941 (+0.0035)
1.6418 (+0.1327)
7.2171 (+0.3359)
6.84
0413
Inaelleaad
QuerRorm
0.72
0296
12
3.55
0.02
Scheddler
Schasller
-B- AsyeDM
DM
AigilM
DM
-Bi
AgyiDM
TMM -B- AngeIM
0.2
u 0.4
0.5
@l
0.6
08
0.9
Figure 8:
As increases, the maximun timestep difference increases, and the alignment first im- proves and then degrades. The extreme differences cause faster-denoisee regions to retain noise for contextual consistency, leading blurry and noisy background in final images (e.g., 0.8, 0.9).
maximum timestep difference can be flexibly controlled. As case study, consider the prompt "a shark riding a bike", and sample 32 images for each value of w to evaluate text-to-image alignment. As shown in Figure [ the results indicate that as w increases (i.e., the maximum timestep difference increases), the alignment first improves and then degrades. The degradation occurs because, when timestep states across pixels differ extremely, the faster denoised regions may be affected by noisy regions, which continue to provide noisy context even at later denoising steps. Consequently, these faster denoised regions tend to preserve a considerable amount noise in order to remain consistent with the context. This effect is particularly evident at = 0.8 and w 0.9, where the generated images exhibit blurry and noisy background regions.
D.3 ABLATION ON DENOISING STEPS
A growing body of work has focused on enabling diffusion models to generate high-quality images with only small number of denoising steps (Xiao et al.) [202 Yin al. 2024). Motivated by this line of research, we further evaluate the performance of AsynDM under different total denoising steps T. Specifically, we set the steps 7 to 5, 10, 20, 30, 40, 50 and 60, and generate 1,280 images for each setting on the animal activity prompt set. The results are summarized in Table 6 Across all denoising-ster configurations, AsynDM consistently improves text-to-image alignment. Figure provides some examples. These results further demonstrate the effectiveness of our method.
20

---

Published as a conference paper at ICLR 2026
Table 6: Alignment performance of AsynDM for different denoising steps T, across prompts
animal activity prompt set. The base model is SD2.1-512-base
Metric
Method
==
T 10
T 20
T 30
T 40
T 50
T 60
DM
0.5924
0.6221
0.6311
0.6330
0.6371
0.6353
0.6346
BERTScore
AsynDM
0.5987
0.6280
0.6364
0.6372
0.6402
0.6414
0.6412
DM
0.3111
0.3574
0.3681
0.3672
0.3689
0.3685
0.3691
CLJPScore
AsynDM
0.3260
0.3636
0.3708
0.3699
0.3729
0.3750
0.3752
DM
-10882
0.0926
0.5801
0.6458
0.7606
0.7543
0.7668
ImageRewardf
AsynDM
-0.7556
0.3087
0.6732
0.7561
0.8692
0,9219
0.9123
DM
2,8703
4.0750
4.7148
4.7359
4,8992
4.9445
4,9382
QwenScoret
AsynDM
3.3164
4.5718
4.9976
5.0218
5.2859
5.5218
5.3234
T-5
7 20
r 30
T=
T=
T= 60
DM
AXyRDM
dog playing basketball
DM
AgyaD
horse cating sandwich
Figure 9: Samples generated by AsynDM compared with DM for different denoising steps T. The base model is SD2.1-512-base.
E MORE SAMPLES
In this section, we present additional samples generated by AsynDM, alongside those from baseline methods. Specifically, Figure 10 presents more samples SD across diverse prompts. Figure [1] presents the samples of the ablation studies Section 5.4 samples SDXL and SD 3.5, respectively.
Figure 12 and Figure [3 present the
F DECLARATION OF LLM USAGE
In preparing this manuscript, used the large language model (LLM) general-purpose writing assistant. Specifically, the LLM was employed to (1) check grammar and correctness the text, and (2) suggest more natural and fluent wording. When using the LLM, first wrote initial draft of the sentence, and then asked the LLM to check and polish it. The LLM did not contribute to research ideas, methods, experiments, results. The authors take full responsibility for the content of this paper.
21

---

Published as a conference paper at ICLR 2026
DM
DM
Z-Sampling
SEG
SCFG
CFG+
AsynDM
concave
a photo of a horse and a giraffe
Three sheep walking together following a trail.
a horse eating sandwich
a photo of book and laptop
A train top of a surfboard.
A zebra underneath a broccoli.
photo of three birds
lizard washing dishes
Figure 10: More samples generated by AsynDM compared with baseline methods. The images sampled by AsynDM show higher text-to-image alignment. The base model used to sample these images SD2.115112base
22

---

Published as a conference paper at ICLR 2026
DM
Prompt: shark riding bike
Prompt:
a chicken eating sandwich
DM
Moneavee concave
AsynDM
+ fixed mask
DM coneave
AsynDM
+fxed mask
Quadratic
Piecewise Lincar
Exponential
DM
DM
Prompt: monkey driving a car
Prompt: a goose playing basketball
DM
AsynDM + +fixed mask
DM concave
AsynDM
fixed mask
Quadratic
Piecewise Linear
Exponential
Figure 11: Samples generated by AsynDM when employing different concave schedulers and using fixed masks. The base model used to sample these images is 1D2.1-512-bbas
23

---

Published as a conference paper at ICLR 2026
DM
DM concave
Z-Sampling
SEG
SCFG
CFG+
AsynDM
a goat eating sandwich
a wolf driving a car
dog playing basketball
goat driving
a lizard washing dishes
a cat eating a sandwich
wolf playing basketball
sheep driving car
Figure 12: Samples generated by AsynDM compared with baseline methods when using SDXL- base-1.0. The images sampled by AsynDM show higher text-to-image alignment.
24

---

Published as a conference paper at ICLR 2026
DM
DM
SEG
SCFG
AsynDM
sheep eating sandwich
a tiger playing basketball
wolf driving car
dolphin washing dishes
fox riding a bike
sheep driving
Figure 13: Samples generated by AsynDM compared with baseline methods when using SD3.5- medium. The images sampled by AsynDM show higher text-to-image alignment.
25


Reviews:
[{"review_id": "4cyWlSsfLa", "reviewer": "Reviewer_QfCK", "summary": "This paper proposes a novel method that utilizes masks (getting from network attention or using fixed masks) to decompose the diffusion time of every pixels in an image, resulting clearer inter-pixel context and significant improvement.", "questions": "Q1:\nThis proposed method uses different time steps for different pixels. However, current research on diffusion focuses on reducing the time step of diffusion. Therefore, when this method is applied to models with small-time diffusion, e.g., T=4 in DDGAN [4], can the effect still be significantly improved?\n\nQ2:\nThis proposed method can be further combined with patchDiff [5] to improve the generation effect?\n\nQ3:\nFrom the Fig.4, we can find there are clear improvement between the proposed methods and other methods. However, the quantitative performance improvement in Table1 are not obvious. I believe this is a question worth explaining.\n\n[4] Xiao Z, Kreis K, Vahdat A. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs[C]//International Conference on Learning Representations.\n\n[5] Wang Z, Jiang Y, Zheng H, et al. Patch diffusion: Faster and more data-efficient training of diffusion models[J]. Advances in neural information processing systems, 2023, 36: 72137-72154.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**S1: Very good innovation.** I believe asynchronous diffusion can bring a lot of inspiration to subsequent work.\n\n**S2: Significant improvement.** Figure 4 (I was looking forward to discovering more in the supplementary materials, but I couldn't find them) and human survey show extremely good improvements.", "weaknesses": "**W1: Claim issue**. In intro., the authors claim that the text-to-image misalignment is caused primarily by synchronous denoising. However, in many other single-step generative models, including GANs and VAEs, the misalignment also is a key problem. Thus, I think this statement lacks strong support. In other words, I believe that the proposed asynchronous denoising method can alleviate this misalignment problem to some extent, but this misalignment may not necessarily be caused by this synchronous denoising. Therefore, I believe that this statement needs to be revised and the author needs to provide a broader discussion on other methods to address this issue (including discussing existing methods [1] [2] [3] in other generative models to address this issue and other potential solutions).\n\n[1] Liao W, Hu K, Yang M Y, et al. Text to image generation with semantic-spatial aware gan[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 18187-18196.\n\n[2] Zhang C, Peng Y. Stacking VAE and GAN for context-aware text-to-image generation[C]//2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM). IEEE, 2018: 1-5.\n\n[3] Wang H, Lin G, Hoi S C H, et al. Cycle-consistent inverse GAN for text-to-image synthesis[C]//Proceedings of the 29th ACM international conference on multimedia. 2021: 630-638.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9GjKTPCXGk", "reviewer": "Reviewer_mDhN", "summary": "The work deals with text-to-image (T2I) diffusion models. The authors argue that the misalignment of the generated image and the textual prompt results from a spatially and temporally uniform (synchronous) change of the denoised image pixels. Hence they propose to denoise some regions differently, at different time steps during the reverse denoising process, the timestep of every pixel being dynamically determined. For this a timestep is assigned to each pixel. The regions of the image that are significantly associated to some tokens of the prompt are thus scheduled according to a specific (decreasing concave) function. The approach is compared to recent baseline models on four prompt datasets, reporting four alignment metrics.", "questions": "- which \"SD 2.1 base\" (line 300) is used ? SD 2.1-512 or SD 2.1-768? \n- why the results on SDXL and SD 3.5  have not been reported in the main paper (and those with SD 2.1 in appendix) ?\n- Could we have more details about the study conducted with humans, in particular on the cohort of 22 participants (see above)?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* the initial motivation -- the fact that it might be worthwhile to denoise differently various regions of an image -- is intuitive and makes sense. It is clearly explained both in the introduction (section 1) and the method itself (section 3).\n\n* the quantitative and qualitative results are reported with three different backbones, including the recent SD 3.5 (in the appendix). The proposed approach is compared to four recent baselines, published in 2024 or 2025 in top-tier venues (CVPR, NeurIPS, ICLR).\n\n* the quantitative results are computed with a significant number of 1280 images per prompt set, with the same random seed for all models.", "weaknesses": "* the paper ignores an important part of the literature relating to generative semantic nursing. Following Attend-and-Excite [d] several papers investigated at optimizing alignment (cross attention) between the prompt and the noise during the backward process e.g Divide and Bind [g] or Syngen [h]. Similarly to the proposed work, these works showed that the \"regions\" of the image are indeed decoded at various timestep, but the conclusion was rather than the diffusion models first reconstruct the high-power, low-frequency components at early denoising stages before adding low-power, high-frequency details at later stages [i,j]. Positioning with regards to these works would thus have been relevant.\n\n* The quantitative results are not convincing\n  - the authors do not adopt previous protocol. In particular for GenEval, they use the same 553 prompt but do not use the metrics of the GenEval benchmark itself, making hard to compare to previous published results. For Drawbench, the metrics are not the same as e.g (Bai LiChen et al, 2025), making hard to compare directly with previous published results\n  - the used BERTscore relies on image description obtained with an ad-hoc model, Qwen2.5-VL-7B-Instruct in this paper. The resulting scores thus evaluates both the models tested and the model used to generate the \"ground truth\". Similar remarks applies to the QwenScore. However, the two other metrics reflects also alignment (see below)\n  - all the metrics deal with text-to-image alignment, ignoring other aspects of image generation. One can understand that this aspect is important to evaluate for this paper, but it should have been asserted that, for example, the image quality is -- at least -- maintained.\n  - the quantitative results are reported without any standard deviation, while the quality of generated images (both for aesthetic and alignment) is known to be variable w.r.t the seed. Given the limited difference in performances in comparison to baselines (in particular ofor the most recent backbones in Table 4 and Table 5), one can have doubt regarding the significance.\n    - by the way, it is hard to understand why the results in the main paper are based on a old model (SD 2.1) while results on more recent SDXL\n\n* the human study is poorly described\n  - there no detail on the 22 participants: are they diversified in gender and age ? Is a majority of them student of the same university as the authors? Or even colleagues? Or the author themselves? Is there some diversity in terms of native language? if so, how the prompt were presented (in English or in their native language) ?\n  - it is not clear how many triplet were shown to the participants, nor how these triplets were chosen. For the automatic evaluation of Table 1 it is said that $4\\times 1280$ images are considered for each of the prompt sets. Just after, on line 376, it is reported that the participant evaluate \"for each group of three candidate\", suggesting that they evaluate 5120 triplets. One can doubt that participant actually made as many evaluation.\n  - there is no inter-annotator agreement, making hard to estimate the relevance of the study\n  - for good practice on human studies, one can refer to [f]\n\n* the qualitative results in Figure 4 are poor for the baselines, but it seems to be mainly due to the old SD 2.1 backbone used. If one uses the [online inference available for SDXL on huggingface](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) -- while it is itself quite old since released in July 2023 -- it is quite easy to get much better results for the base DM than those reported in Figure 4 (with the same prompts).\n  - in the Appendix, on Figure 10 and 11, the qualitative results for SDXL and SD 3.5 looks often better for the baselines than the proposed model.\n\n* minor\n  - the definition of $x_i\\in\\mathbb{R}^{n_c\\times h\\times w}$ on line 194 should be introduced earlier, before equation (4) around line 184. It is indeed a crucial change for the proposed method since it reflects the *pixel-level* aspect.\n  - the references for \"text-to-image misalignment\" on lines 057-058 are recent but inappropriate since this phenomenon has been identified well before for diffusion models, e.g in DALL-e [a] released as a preprint, Imagen (Saharia et al, 2022), DAAM [b], Structured Guidance [c] and Attend&Excite [d]. Not to mention previous works with GANs e.g SOA [e]. The reference (Liu et al, 2025) may be relevant since it is a review, although it seems to be only a preprint and not (yet?) published. However (Hu et al, 2025a) is just a recent paper dealing with a problem already known.\n  - which \"SD 2.1 base\" (line 300) is used ? SD 2.1-512 or SD 2.1-768? \n\n[a] A. Ramesh et al. \"Hierarchical Text-Conditional Image Generation with CLIP Latents\". In: arXiv 2204.06125 (2022). arXiv: 2204.0612\n\n[b] R. Tang et al. \"What the DAAM: Interpreting Stable Diffusion Using Cross Attention\". ACL 2023.\n\n[c] W. Feng et al. \"Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis\". ICLR 2023.\n\n[d] H. Chefer et al. \"Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models\". In: ACM Trans. Graph. 42.4 (July 2023)\n\n[e] Hinz et al \"Semantic Object Accuracy for Generative Text-to-Image Synthesis\", TPAMI 2022 (and arXiv:1910.13321 in 2019)\n\n[f] M. Otani et al. \"Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation\". CVPR, 2023 \n\n[g] Li et al \"Divide & Bind Your Attention for Improved Generative Semantic Nursing\", BMVC 2023\n\n[h] Rassin et al \"Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment\" ,NeurIPS 2023\n\n[i] S. Rissanen, M. Heinonen, and A. Solin. “Generative modelling with inverse heat dissipation”. ICLR 2023\n\n[j] Y.-H. Park et al. “Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry”. NeurIPS 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sQjnWUNc2M", "reviewer": "Reviewer_6tYP", "summary": "This paper proposes an asynchronous diffusion framework for text-to-image generation. The core idea is to create a pixel-level timestep scheduler and let prompt-related regions be decoded more slowly. Extensive experiments demonstrate AyncDM achieves better performance among other training-free text-to-image alignment approaches.", "questions": "1. How do the authors address the training–inference gap in AsynDM? Is there any theoretical or empirical evidence showing that this discrepancy can be safely ignored, or that it does not occur during sampling?\n2. How does AsynDM apply or adapt the distracted cross-attention mechanism to SD3.5-Medium or other architectures based on MMDiT? The process seems non-trivial and could benefit from further clarification.\n\nI would be happy to raise my score if these concerns are properly addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The idea is novel and well-motivated. The proposed approach, using cross-attention as a mask indicator, is quite intuitive and easy to follow.\n2. This paper is clearly written and well-organized.\n3. The comparison results are very promising, showing clear advantages over relevant baselines.\n4. The authors conduct comprehensive experiments across multiple model baselines, sampler choices, and ablation settings, which provide strong and convincing evidence for the proposed approach.", "weaknesses": "1. My main concern is the inconsistency between the training and inference stages of AyncDM. From my understanding, during training, noise is added synchronously into all pixels, and the diffusion model predicts $f_\\theta(x_t)$ where $x_t$ is a noised latent with uniform noise levels. However, during inference, the input of the diffusion model is asynchronous or spatially varying noise levels across pixels. How can the model reliably decode latents that contain uneven noise distributions, given that it was never explicitly trained under such conditions?  Some marginal artifacts may occur in these scenarios. \n2. The distracted attention mask relies heavily on cross-attention maps. However, in the early denoising steps, cross-attention maps are often noisy and unstable, which may lead to unreliable or ambiguous guidance when determining which regions should be denoised faster or slower.\n3. Similarly, for more advanced T2I models such as SD3.5 or Flux that adopt the MMDiT framework instead of conventional cross-attention, deriving reliable spatial masks becomes more challenging, since text and image latents are concatenated within a self-attention module.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)