Title: CPO: Condition Preference Optimization for Controllable Image Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Models

Controllable Image Generation

ControlNet
Abstract: To enhance controllability in text-to-image generation, ControlNet introduces image-based control signals, while ControlNet++ improves pixel-level cycle consistency between generated images and the input control signal. To avoid the prohibitive cost of back-propagating through the sampling process, ControlNet++ optimizes only low-noise timesteps (e.g., $t < 200$) using a single-step approximation, which not only ignores the contribution of high-noise timesteps but also introduces additional approximation errors. A straightforward alternative for optimizing controllability across all timesteps is Direct Preference Optimization (DPO), a fine-tuning method that increases model preference for more controllable images ($I^{w}$) over less controllable ones ($I^{l}$). However, due to uncertainty in generative models, it is difficult to ensure that win--lose image pairs differ only in controllability while keeping other factors, such as image quality, fixed. To address this, we propose performing preference learning over control conditions rather than generated images. Specifically, we construct winning and losing control signals, $\mathbf{c}^{w}$ and $\mathbf{c}^{l}$, and train the model to prefer $\mathbf{c}^{w}$. This method, which we term \textit{Condition Preference Optimization} (CPO), eliminates confounding factors and yields a low-variance training objective. Our approach theoretically exhibits lower contrastive loss variance than DPO and empirically achieves superior results. Moreover, CPO requires less computation and storage for dataset curation. Extensive experiments show that CPO significantly improves controllability over the state-of-the-art ControlNet++ across multiple control types: over $10\%$ error rate reduction in segmentation, $70$--$80\%$ in human pose, and consistent $2$--$5\%$ reductions in edge and depth maps. The error rate is defined as the difference between the evaluated controllability and the oracle results. Our project is available \textcolor{blue}{\href{https://zonglinl.github.io/CPO_page}{here}}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes using reinforcement learning for ControlNet-based image generation. Prior work such as ControlNet++ optimizes only low-noise timesteps, which can introduce optimization errors. While Direct Preference Optimization (DPO) can optimize across all timesteps, it lacks fine-grained control, making it difficult to improve controllability without affecting other generation factors. To address this, the authors introduce a novel Condition Preference Optimization approach that constructs winning and losing control signals and trains the model to prefer the winning ones. Experimental results demonstrate superior performance over both DPO and ControlNet++ across various control types and evaluation metrics.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The proposed idea is novel and well-motivated. To the best of my knowledge, no prior work has attempted to apply the DPO method for conditional image generation following ControlNet. The authors also identify the challenges in directly applying DPO and introduce a new approach—Conditional Preference Optimization (CPO)—to address these issues.\n\n2. The paper is generally well-written and easy to follow.\n\n3. The proposed method achieves strong controllability scores, demonstrating the effectiveness of CPO. Additionally, the authors conduct thorough ablation studies comparing CPO with DPO and analyzing hyperparameter choices, which further strengthen the experimental results.\n\n\nWeaknesses\n1. Despite the novelty of the approach, it raises questions regarding the theoretical equivalence between the proposed CPO and the original DPO. As stated in the introduction, while the relationship between the control and the generated images may be bidirectional, directly switching to CPO changes the data distribution compared to DPO. If CPO is not strictly equivalent, some clarification or refinement in the introduction would be helpful to better explain the motivation and implications.\n\n2. In practice, when applying perturbations, how can we ensure that the perturbed control signals extracted from generated images share a similar distribution with real control signals? The paper mentions that the perturbations are derived from the generated images—but are they of comparable quality to the original control signals? Why not directly perturb the original control inputs instead?\n\n3. The authors should consider conducting a user study to better evaluate the results. Human judgment can provide a more reliable assessment of how well the generated images align with the control inputs, compared to simply comparing the extracted control signals from generated images. Based on the current results, the quality of the extracted control signals is highly dependent on the extraction method used. For instance, the depth map for ControlAR in Figure 4 does not appear meaningful. Additionally, the HED results represent particularly challenging cases, and I would not conclude that ControlAR performs worse than the proposed method in those examples. Furthermore, in Figure 5(a), the alignment between the human subject and the control input is not very accurate.\n\nQUESTIONS:\nWhile the method shows good controllability scores, it sometimes yields lower FID and CLIP scores. It would be helpful if the authors could discuss this trade-off and provide insights into how controllability affects image quality and alignment with the text.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

comments: SUMMARY:
This paper aims to enhance the controllability of image generation using ControlNet. It begins by examining the shortcomings of existing methods based on Direct Preference Optimization, which often struggle with uncertainties inherent in generative models—specifically, a winning image may compromise on other factors such as overall image quality. To address these challenges, this paper proposes Conditioned Preference Optimization, which constructs preference condition pairs rather than relying solely on image pairs. The proposed method demonstrates superior performance compared to previous approaches.

STRENGTHS AND WEAKNESSES:
Strength:

The proposed method has clear motivation and analysis, providing analysis for previous methods.

The proposed method is simple but effective, with similar formulation to previous methods.

The paper also provides theoretical proof for the proposed method, lending it greater validity.

This paper is well-written. For example, the Fig.1 and Fig.2 can clear illustrate the motivation and strategy.

Weaknesses:

For the condition pair, how to control the differences? If the difference is very large, it might be invalid. The authors should detail the process of constructing the pair, such as pose, canny, depth, segmentation ,etc.

The backbone is still SD1.5. To better validate its effectiveness, please use SDXL, even SD3 and FLUX.

QUESTIONS:
For the condition pair, how to control the differences? If the difference is very large, it might be invalid. The authors should detail the process of constructing the pair, such as pose, canny, depth, segmentation ,etc.

The backbone is still SD1.5. To better validate its effectiveness, please use SDXL, even SD3 and FLUX.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new Condition Preference Optimization (CPO) method that aligns the conditions directly with DPO instead of using images for controllable image generation. It uses the original condition as the winning condition while treating the generated condition as the losing condition. Experiments are conducted on segmentation, depth, canny, edge and pose conditions across 5 different benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**: The paper proposes a novel method for improving the alignment and controllability in conditional image generation with the CPO method that uses conditions as preference images instead of their RGB counterparts. Experimental results show improved performance over the baseline ControlNet or ControlNet++ across pose, depth, segmentation and sketch conditions.\n\n**Weaknesses**: \nThe proposed method uses a discriminative model (condition detector) to obtain the losing conditions from the generated images of the baseline model. However, this assumes that the detector is perfect which is not the case in practice. This results in error propagation to the CPO where even good images will be rejected. So, it is sensitive to the detector used. How does the performance vary with different detectors? How many images are sampled to obtain the losing condition? Is it just one?\n\nThe proposed method requires a strong pretrained model as initialization such as the ControlNet or ControlNet++ and so the computational complexity increases. How does the performance compare to training baseline models for longer (same amount of time as CPO) with the new data? Is the improvement solely because of CPO?\n\n The performance improvements when evaluated under the same CFG configuration of 7.5 listed in Tables 1 and 2 are not very significant compared to the additional complexity involved in training. What is the performance difference between the methods when using the CFG scale of 4.0? It is not clear why the performance changes a lot at CFG 7.5 especially since the initialization is from a base model that uses a CFG of 7.5.\n\nWhat is the effect of this alignment on the background regions? The model’s performance is specifically optimized to improve the controllability in the regions specified by the conditions. Does it degrade the performance for those regions outside the conditions? This can also be seen from the FID scores reported in Table 2 where the performance is comparable or slightly inferior to the baseline models.\n\nIn practice, we require both controllability and high-quality images. So, preference along both dimensions is essential. Will there be any interference when CPO is performed on top of DPO or vice-versa?\n\nQUESTIONS:\nHow does the performance vary with different detectors? How many images are sampled to obtain the losing condition? Is it just one?\n\nHow does the performance compare to training baseline models for longer (same amount of time as CPO) with the new data? Is the improvement solely because of CPO?\n\nWhat is the effect of this alignment on the background regions? The model’s performance is specifically optimized to improve the controllability in the regions specified by the conditions. Does it degrade the performance for those regions outside the conditions?\n\nWill there be any interference when CPO is performed on top of DPO or vice-versa?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes the CPO (Condition Preference Optimization) method, which can improve the compliance of controllable image generation with control signals. The CPO method trains the model to prefer winning control signals by generating both winning and losing control signals. Compared to DPO, it requires significantly fewer computation for training data generation, making it more efficient. Compared to SoTA methods, it achieves better controllability. Under control signals such as segmentation and pose, it demonstrates a notable reduction in errors.\n\nSTRENGTHS AND WEAKNESSES:\nstrength\n- The paper demonstrates a clear improvement in controllability, such as achieving  10%+ error rate reduction in segmentation, 70–80% error rate reduction in human pose, approximately 2–5% reduction on edges and depths after fine-tuning. This error reduction is relative to the model before fine-tuning.\n- The writing of the paper is good, and it is quite readable. There is theoretical proof that the variance of CPO is smaller than that of DPO.\n- The motivation of the paper is clear. The method is significantly more efficient than DPO, and easier to generate training data.\n\nweakness\n- The number of images for qualitative evaluation is too limited. In Section G, the focus is mainly on the generation results of this method, lacking more qualitative comparisons with other methods. The evaluation results are primarily based on the quantitative outcomes of  algorithms, without comparative assessment through a user study.\n- There is a risk that training is favorable for a specific detection model, rather than improving the image information. The model used to generate data is the same as the one used for evaluation, such as YOLO-11x-Pose for pose estimation.\n\nQUESTIONS:\n- Is the improvement related to the correlation between the detector model for data generation and evaluation? If a different detector model is used during evaluation, will the same improvement still be achieved?  Taking Pose condition as an example, the CPO training data was generated using YOLO-11x-Pose, and the mAP calculation also used YOLO-11x-Pose. Could it be possible that visually, the pose of a person in the image appears incorrect, but YOLO-11x-Pose considers the pose more consistent? For the Pose mAP evaluation in Table 1, could another pose estimation model be used for assessment?\n- The comparison with DPO is not sufficiently thorough. Could you provide more qualitative results?  \n- Could you also provide more qualitative comparative results?\n\nLIMITATIONS:\nThe authors adequately addressed the limitations and potential negative societal impact of their work. \"yes\".", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}