Title: Free-Lunch Color-Texture Disentanglement for Stylized Image Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion models

style transfer

disentangled stylized image generation
Abstract: Recent advances in Text-to-Image (T2I) diffusion models have transformed image generation, enabling significant progress in stylized generation using only a few style reference images. However, current diffusion-based methods struggle with \textit{fine-grained} style customization due to challenges in controlling multiple style attributes, such as color and texture. This paper introduces the first tuning-free approach to achieve free-lunch color-texture disentanglement in stylized T2I generation, addressing the need for independently controlled style elements for the Disentangled Stylized Image Generation (DisIG) problem. Our approach leverages the \textit{Image-Prompt Additivity} property in the CLIP image embedding space to develop techniques for separating and extracting Color-Texture Embeddings (CTE) from individual color and texture reference images. To ensure that the color palette of the generated image aligns closely with the color reference, we apply a whitening and coloring transformation to enhance color consistency.  Additionally, to prevent texture loss due to the signal-leak bias inherent in diffusion training, we introduce a noise term that preserves textural fidelity during the Regularized Whitening and Coloring Transformation (RegWCT). Through these methods, our Style Attributes Disentanglement approach (SADis) delivers a more precise and customizable solution for stylized image generation. Experiments on images from the WikiArt and StyleDrop datasets demonstrate that, both qualitatively and quantitatively, SADis surpasses state-of-the-art stylization methods in the DisIG task.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel approach called Style Attribute Disentanglement (SADis), designed to address a key challenge in stylized image generation: achieving independent control over color and texture. Specifically, SADis leverages the additive property of image prompts to disentangle color and texture for the first time without any training. This enables users to independently extract color and texture information from different reference images.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1、The method is easy to follow. This paper proposes using the CLIP model to extract disentangled color and texture features from reference images and guide the generation of target images through the IPA plugin. Both qualitative analysis and quantitative metrics demonstrate strong performance.\n\n2、It requires no additional data training, can leverage any existing T2I model, and is fully compatible with the SD community ecosystem.\n\nWeaknesses\n\n1、The ablation study in Table 2 suggests that the proposed component contributes primarily to the injection of style color information, with limited impact on texture preservation. This raises the question of whether the texture-related performance gain is mainly attributable to the IPA module, rather than the disentangled texture features themselves.\n\n2、The original IPA module applies cross-attention across multiple decoder layers, whereas the proposed method restricts injection to only the first decoder layer. It remains unclear whether this simplification affects the integrity of the generation process, potentially limiting the model’s ability to integrate style features progressively.\n\n3、The ablation study does not sufficiently isolate the effect of the proposed TextureExtraction module. It is therefore difficult to determine whether improvements in texture retention are due to this module or the inherent capabilities of IPA. More targeted experiments would help clarify this.\n\n4、How significant is the impact of the singular value modulation parameters γ (gamma) and β (beta) on the generation process? How are they set during training, and are they learnable.\n\nQUESTIONS:\nMy major concerns are the weakness. I would like the authors to directly address these concerns in their response. My final rating is subject to change based on the quality and clarity of the authors’ feedback.\n\nLIMITATIONS:\nThe authors are encouraged to answer the questions and address the weakness above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the additivity of the CLIP image embedding, especially for extracting color information of arbitrary images. Based on the observation, the authors propose a novel method, namely SADis, that disentangles the learning of color and texture to represent style characteristics. To remove the redundant color information from the texture image, several strategies are introduced, manipulating image embeddings by average operation, SVD, and WCT. Experiments show the disentanglement of color and texture benefits stylized image generation, as well as in-depth investigations on a variety of possible applications.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**: \n\nThe paper is well-written and easy to follow. The phenomenon of image-prompt additivity is very intuitive, and additional analysis in the Appendix is also interesting. I really like these illustrations, which help me to grasp the idea quickly. The final method is well motivated based on the observations. The evaluation is thorough and demonstrates the effectiveness of the proposed method very well. I also appreciate the human judgment and time cost reported.\n\n**Weaknesses**:\n\nWhile the method clearly surpasses existing style transfer methods in the color-texture-content reference scenario, the authors use LLM-generated descriptions during comparison. This setting might be unfair for other approaches.\n\nQUESTIONS:\n1. The setting of \"Baseline\" in Fig. 4 (a) is not clear. I found it in lines 175-177 as \"using only the grayscale texture embedding\". Does this mean the texture branch w/o averaged embedding, SVD, and RegWCT?\n\n2. Since previous methods lack disentanglement ability, why not use the same style image for both the color and texture references to SADis, and then compare with these methods? In my opinion this would be fairer than using generated captions. In this case, I am curious about the difference between SADis, IP-Adapter, and InstantStyle, as the other two methods seem to capture texture very well but are inferior in color (observe from Fig. 5 and  user study in Tab. 1) \n\n3. Ablation studies are only conducted on two components, i.e., SVD and RegWCT. I found these two components contribute more on the color consistency in Tab. 2. It seems that they help remove color information from the texture image to inject color information, but I guess the learning of texture might benefit from another design, e.g., the concatenated averaged embedding, in which an ablation study is missing. \n\nMinor suggestions: \n1. Fig.4 better provides the text prompts or the content images (if content-conditioned generation using ControlNet) for the generated images.\n\n2. Should the line 177 be \"Fig. 4 (a)-2nd column\", line 193 be \"Fig. 4 (a)-3rd column\", and line 226 be \"Fig. 4 (a)-4th column\"?\n\n3. Fig. 4 (b) the description \"+z_T injection\", if it is the formulation in line 217, I recommend rephrasing it into noise injection, or directly RegWCT, as T is the starting point, but this injection seems to perform in a period.\n\nLIMITATIONS:\nThe authors describe the limitations of the proposed SADis method in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-26"}

comments: SUMMARY:
This paper introduces a tuning-free method of disentangling color and texture for stylized image generation. The authors discover that the color embedding and textural embedding can be added as the style condition. The authors propose some techniques to improve generation quality, such as SVD embedding optimization and regularized whitening-coloring transforms.

STRENGTHS AND WEAKNESSES:
**Strengths**

- The idea of disentangling color and texture information addresses a practical need.
- The proposed method is tuning-free, making it easily applicable to various backbones such as IP-Adapter and ControlNet.
- The design choice of using additive CLIP embeddings, along with further optimizations via SVD or WCT, is reasonable.
- The generated results are of good quality.

**Weaknesses**

- From Table 2, SVD improves color persistence but worsens texture, especially when combined with RegWCT.
- The authors propose various techniques, such as SVD, WCT, and noise regularization. However, all of these methods aim to strike a better trade-off between color and texture quality, and no single component is able to improve both aspects simultaneously.
- This method relies on the additive property of CLIP as a vision encoder, but its effectiveness with other encoders, such as T5 or DINO, has not been validated.

QUESTIONS:
Please see weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

comments: SUMMARY:
The author proposes a tuning-free style-preserving scheme for texture and color disentanglement. Leveraging the additive property of the CLIP embedding space and assuming that image style features can be decomposed into color and texture features as a strong hypothesis, the disentanglement scheme is designed. Different from existing methods that require model fine-tuning, SADis achieves style disentanglement solely through forward propagation and feature transformation, without modifying the parameters of pre-trained diffusion models, thus significantly reducing computational costs.

STRENGTHS AND WEAKNESSES:
Strength:

1. The writing of the paper is clear.

2.The motivation of the paper is clear.

3. The method proposed in the paper is interesting, which decouples CLIP features to obtain texture and color information. However, I still worry that such information is unstable.

4. The paper provides some quantitative and user studies.

Weaknesses:

1.Many contents of the paper are based on some strong assumptions, which may lack sufficient theoretical basis.

2.Lack of verification with some complex prompts.

3.The visualization results in Figure 5 do not demonstrate that the proposed method has significant advantages.

QUESTIONS:
1.How to verify the correctness of the strong hypothesis in the process of Eq1?

2.The author demonstrates some simple prompt scenarios, but lacks verification with complex prompts.

3.How to verify the correctness of the strong hypothesis in the process of Eq3?

4.I'm more concerned about the issue of stability. Will there be significant differences when generating multiple images (rolling).

5.Some recent methods only support single-image processing. How to perform inference for multiple images?

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03