Title: Aligning Text to Image in Diffusion Models is Easier Than You Think
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion model

flow model

contrastive learning

representation alignment

soft token
Abstract: While recent advancements in generative modeling have significantly improved text-image alignment, some residual misalignment between text and image representations still remains. Some approaches address this issue by fine-tuning models in terms of preference optimization,  etc., which require tailored datasets. Orthogonal to these methods, we revisit the challenge from the perspective of representation alignment—an approach that has gained popularity with the success of REPresentation Alignment (REPA). We first argue that conventional text-to-image (T2I) diffusion models, typically trained on paired image and text data (i.e., positive pairs) by minimizing score matching or flow matching losses, is suboptimal from the standpoint of representation alignment. Instead, a better alignment can be achieved through contrastive learning that leverages existing dataset as both positive and negative pairs. To enable efficient alignment with pretrained models, we propose SoftREPA—a lightweight contrastive fine-tuning strategy that leverages soft text tokens for representation alignment. This approach improves alignment with minimal computational overhead by adding fewer than 1M trainable parameters to the pretrained model. Our theoretical analysis demonstrates that our method explicitly increases the mutual information between text and image representations, leading to enhanced semantic consistency. Experimental results across text-to-image generation and text-guided image editing tasks validate the effectiveness of our approach in improving the semantic consistency of T2I generative models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tries to improve the text-image alignment in diffusion-based T2I model. Following REPresentation Alignment(REPA), the authors introduce contrastive representation loss to diffusion model training. The authors define the similarity measure using the the variation of the denoising score matching loss and introduce extra soft tokens at each layers. The experimental results show improvement of the finetuned model over the baseline model.\n\nSTRENGTHS AND WEAKNESSES:\n***Strengths***\n\nThe idea of introducing contrastive representation loss makes senses. And the experimental results suggest improvement over the baseline model. \n\n\n***Weaknesses***\n\na. The main idea of introducing contrastive representation directly follow REPresentation Alignment(REPA) [1]. And implementation-wise,  introducing and optimizing some special text token are also not new in diffusion model (e.g. [2][3]) \n\nb. The design of the similarity measure is new for this paper. However, I have some concerns regarding this design choice:\n\n1. The original REPA [1] work introduces the pretrained self-supervised visual representations to calculate the similarity. Yet this work purely relies on the diffusion model itself to define the similarity. Using pretrained self-supervised visual representations make more sense to me as the pretrained model brings extra knowledge during the training. It might be better if the authors can experimentally compare their design with the one using pretrained self-supervised visual representations (i.e. original REPA).\n\n2. Also, the authors mainly compare their finetuned model with the unfinetuned baseline model. I think it is worth adding an ablation which keeps the structure design of the soft-token, but finetune the model with the original diffusion loss and same data. This mainly eliminates the possibility that the improvement comes from continue training on high quality data and can demonstrate that contrastive learning really plays an important role here.\n\n3. This design introducing several new hyper-parameters, including the number of soft tokens, the time scheduling parameter, the layer to include the soft tokens. Also, in appendix B, it seems that for different models, one might need to decide whether to include original denoising loss. These make me concern about the generalizability of the proposed design.\n\n[1] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think\n[2] Null-text Inversion for Editing Real Images using Guided Diffusion Models\n[3] DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation\n\nQUESTIONS:\n1. According to equation 10 and 11, the similarity measure is defined as the variance of denoising score matching loss. As an expectation calculation is included in the equation, do the authors estimate the expectation with Monte Carlo method? If so, how many samples are needed for this estimation for one text-image pair? And will the number of samples (using in Monte Carlo estimation) influenced the performance? When doing the estimation, what is the distribution for timestep sampling?\n\n2. What is the \"time scheduling parameter\" $\\tau(t)$ used in the paper? Are the experimental results sensitive to the choice of $\\tau(t)$?\n\n3. What data source do the author use to train the model?\n\nLIMITATIONS:\nYes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-09"}

comments: SUMMARY:
The paper introduces SoftREPA; a novel and lightweight fine-tuning method for improving t2i alignment in pre-trained t2i diffusion models. The method freezes the pre-trained models and introduces a small number of learnable "soft tokens". These learnable tokens are optimized with contrastive loss that leverages the denoising objective to better differentiate between positive and negative text-image pairs. The authors provide theoretical analysis by showing that this approach is related to maximizing the mutual information between the text and image pairs. The authors demonstrated empirically an enhanced semantic consistency for both image generation and editing tasks.

STRENGTHS AND WEAKNESSES:
Strengths

* Pluggable light-weight module to pretrained t2i models which can used to improve t2i alignment with light-weight finetuning.
* Repurposing the flow matching network output for contrastive learning.
* Provide convincing theoretical analysis that SoftREPA is implicitly maximizes the mutual information between text-image pairs.

Weaknesses

* The paper didn't discuss how the choice of positive/negative pairs will impact the learning, not how to select a finetuning dataset.
* The paper didn't provide convincing comparisons to relavant techniques, instead they compared to RankDPO which is orthogonal method.
* Table 1. shows significant drop in counting metric.

QUESTIONS:
As discussed in the weaknesses above.

1. How would you choose a finetuning dataset especially that Table 1. shows significant drop in counting.
2. How would you better compare this method to other relevant methods? For instance, the author mentioned several training-free techniques (like CFG++) which could be potential candidates to compare to.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors approach the task of improving text image alignment in generative models. The paper proposes a method termed SoftREPA that attempts to align the features using a contrastive objective. \nMore concretely, the authors introduce “soft text tokens” that are concatenated to the regular text tokens, and fine tuned using a contrastive objective, where the positive pair corresponds to the aligned text image pair, and the negative samples correspond to unaligned text-image pairs. The soft text tokens vary at different layers and different denoising steps, and are the only learnable parameter, the rest of the model is kept frozen. \nThe authors also show that minimizing this contrastive objective resembles maximizing the mutual information between the text and image pairs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The contrastive loss to align the features is well motivated and the idea of using learnable soft text tokens is interesting and unique, that I have not seen in prior papers. \n- The method proposed in this paper is quite general, and the authors demonstrate this by applying it to many different base model architectures (SD1.5, SDXL, SD3), and two tasks (text-to-image and image editing). \n\nWeaknesses:\n- Importance of the soft text tokens. It is unclear to me whether the improvement shown in the paper is primarily due to the contrastive objective or because of the use of soft text tokens. It is unclear if you would get similar, better, or worse results if you simply finetune the generative models with a small LoRA. \n\n- Evaluation of text-to-image generation. I have a couple of doubts based on the quantitative results shown in table 1. First is that the authors only compare to the pre-trained non-finetuned model for evaluating the human preference, text alignment, and image quality. Even in this limited comparison, it seems like the base model has higher image quality. \nSecond, how is LPIPS used to evaluate the image quality? LPIPS is a network that is used to compare two images. It is unclear to me how this can be used to evaluate the quality of a text-to-image model. Perhaps the authors can add some implementation details about this metric. \n\n- The authors discuss several other related works that attempt this task of improving the text-image alignment (cfg++, attend-and-excite, DOODL, DiffusionDPO, …). However the authors do not compare to any of these prior works in Table 1. It would be useful to see how the proposed method compares to other methods in the literature.\n\nQUESTIONS:\nMy questions for the authors mostly stem from the weaknesses i listed above. \n- could the authors comment on the need for / importance of the soft text tokens.\n- How is the LPIPS metric implemented for measuring the quality of text-to-image generation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work takes a different approach by focusing on representation alignment, inspired by the success of REPA. The authors propose SoftREPA, a lightweight contrastive fine-tuning method that uses soft text tokens to improve alignment with minimal computational cost (less than 1M additional parameters). Theoretical analysis shows it enhances mutual information between text and image features, leading to better semantic consistency. Experiments confirm its effectiveness in both text-to-image generation and editing tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed SoftREPA is a lightweight contrastive fine-tuning method that introduces fewer than 1M additional trainable parameters, making it highly efficient and easy to integrate with existing pretrained models.\n2. The method is backed by a solid theoretical analysis showing that it explicitly increases mutual information between text and image representations, which leads to improved semantic consistency.\n3. The approach is evaluated on both text-to-image generation and text-guided image editing tasks, demonstrating consistent improvements in semantic alignment and task performance across multiple benchmarks.\n\nWeaknesses:\n1. The paper lacks sufficient comparison between the proposed Soft Tokens and other fine-tuning techniques like LoRA. Otherwise, the authors should provide more explanations on why Soft Tokens are more suitable for the settings.\n2. The qualitative and quantitative results seem to be contradictory with each other. For example, according to the Table 1, the performance of SD3 in Counting is better than SD3 + Ours. However, the conclusion is not supported by the results of 'Three zebras' in Figure 3.\n\nQUESTIONS:\n1. The logic chain from SoftREPA to Soft Tokens is not clearly demonstrated in the paper. Why use Soft Tokens instead of other finetuning techniques like LoRA? Furthermore, LoRA does not put more burden upon time cost by merging them into the model.\nPlease also refer to Weaknesses part.\n\nLIMITATIONS:\nPlease refer to Weaknesses part.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SoftREPA, a novel and lightweight method to significantly improve text-to-image alignment in diffusion models. The authors argue that conventional training is suboptimal for representation alignment and instead propose a contrastive learning framework that trains a small set of learnable \"soft tokens\" (fewer than 1M parameters) to better distinguish between matching and non-matching text-image pairs, while keeping the base model's weights frozen. This approach, theoretically grounded in maximizing mutual information, is shown to be highly effective and efficient, enhancing prompt fidelity for both image generation and text-guided editing across various models like Stable Diffusion 1.5, XL, and 3, with negligible additional computational overhead.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The approach demonstrates significant and broadly generalizable performance gains across various major diffusion models and tasks.\n2. Its effectiveness is validated through comprehensive experiments using a wide range of metrics, including human preference scores and detailed ablation studies.\n3. The core concepts are presented with exceptional clarity, aided by intuitive figures that simplify complex mechanisms for the reader.\n\nWeaknesses:\n1. The main weakness is that the two components are not strongly related. The author should first test fine-tuning with LORA to better illustrate the usefulness of contrastive loss, as the soft tokens are relatively independent.\n2. This method may slightly reduce perceptual image quality metrics like FID and LPIPS because it prioritizes stricter adherence to text.\n3. It can introduce new failure modes, such as decreased counting accuracy, by over-interpreting alignment cues in text prompts.\n4. The title \"Easier Than You Think\" is somewhat exaggerated, as implementing the method still requires a significant contrastive training setup on large datasets.\n\nQUESTIONS:\n1. To better isolate the contribution of the contrastive loss from the soft token architecture, have you considered applying the same loss function with an alternative parameter-efficient tuning method like LoRA?\n2. How does it perform on prompts requiring complex compositional understanding, such as intricate spatial relationships (e.g., \"a red cube on top of a blue sphere to the left of a green pyramid\") or abstract concepts?\n3. The paper positions SoftREPA as an alternative to preference-tuning methods like DPO [1] and DRTune [2]. Could these two approaches be complementary? \n4. Did the authors explore the impact of different negative sampling strategies, such as using \"hard negatives\" (e.g., pairing an image with a very similar but incorrect text prompt) versus the current batch-based random sampling?\n\n[1] Wallace B, Dang M, Rafailov R, et al. Diffusion model alignment using direct preference optimization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 8228-8238.\n[2] Wu X, Hao Y, Zhang M, et al. Deep reward supervisions for tuning text-to-image diffusion models[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 108-124.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-07-08"}