NeurIPS 2025

Diffusion Adaptive Text Embedding for
Text-to-Image Diffusion Models
Byeonghu Na1
Minsang Park1
Gyuwon Sim1
Donghyeok Shin1
HeeSun Bae1
Mina Kang1
Se Jung Kwon2
Wanmo Kang1
Il-Chul Moon1,3
1KAIST, 2NAVER Cloud, 3summary.ai
{byeonghu.na,pagemu,gkwlaks4886,tlsehdgur0,cat2507,kasong13}@kaist.ac.kr,
sejung.kwon@navercorp.com , {wanmo.kang,icmoon}@kaist.ac.kr
Abstract
Text-to-image diffusion models rely on text embeddings from a pre-trained text
encoder, but these embeddings remain fixed across all diffusion timesteps, limiting
their adaptability to the generative process. We propose Diffusion Adaptive Text
Embedding (DATE), which dynamically updates text embeddings at each diffusion
timestep based on intermediate perturbed data. We formulate an optimization
problem and derive an update rule that refines the text embeddings at each sampling
step to improve alignment and preference between the mean predicted image and
the text. This allows DATE to dynamically adapts the text conditions to the
reverse-diffused images throughout diffusion sampling without requiring additional
model training. Through theoretical analysis and empirical results, we show that
DATE maintains the generative capability of the model while providing superior
text-image alignment over fixed text embeddings across various tasks, including
multi-concept generation and text-guided image editing. Our code is available at
https://github.com/aailab-kaist/DATE.
1
Introduction
Text-to-image generation has recently received significant attention due to its capability to generate
realistic and semantically accurate images from textual prompts. This progress has been largely driven
by diffusion models [18, 53], particularly with large-scale models such as DALL·E [45] and Stable
Diffusion [47]. These models use pre-trained text encoders like CLIP [42] and T5 [43] to encode
prompts into embeddings, providing crucial semantic information to diffusion models. Notably, the
quality and semantic alignment of the gener...