RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models

Authors: xinchen  zhang
ling  yang
yaqi  cai
zhaochen  yu
kai-ni  wang
jiake  xie
ye  tian
minkai  xu
yong  tang
yujiu  yang
bin  cui
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 10:27:41

Abstract: Diffusion models have achieved remarkable advancements in text-to-image generation. However, existing models still have many difficulties when faced with multiple-object compositional generation. In this paper, we propose ***RealCompo***, a new *training-free* and *transferred-friendly* text-to-image generation framework, which aims to leverage the respective advantages of text-to-image models and spatial-aware image diffusion models (e.g., layout, keypoints and segmentation maps) to enhance both realism and compositionality of the generated images. An intuitive and novel *balancer* is proposed to dynamically balance the strengths of the two models in denoising process, allowing plug-and-play use of any model without extra training. Extensive experiments show that our RealCompo consistently outperforms state-of-the-art text-to-image models and spatial-aware image diffusion models in multiple-object compositional generation while keeping satisfactory realism and compositionality of the generated images. Notably, our RealCompo can be seamlessly extended with a wide range of spatial-aware image diffusion models and stylized diffusion models. Code is available at: https://github.com/YangLing0818/RealCompo

Main paper: 
 introduction: 
 The field of diffusion models has witnessed exciting developments and significant advancements recently [65, 46, 19, 45, 40, 73] . Among various generative tasks, text-to-image (T2I) generation [33, 20, 64] has gained considerable interest within the community. T2I diffusion models such as Stable Diffusion [41] , Imagen [42] and DALL-E 2/3 [39, 4] have exhibited powerful capabilities in generating images with high aesthetic quality and realism [4, 36] . However, they often struggle to align accurately with the compositional prompt when it involves multiple objects or complex relationships [28, 3, 34] , which requires the model to have strong spatial-aware ability.One potential solution to optimize the compositionality of generated images is providing a spatialaware condition to control diffusion models [12, 66, 58] , such as layout/boxes [35, 14] , keypoint/pose [72] and segmentation map [22] . These spatial-aware conditions are fundamentally similar in functioning, thus we mainly focus our analysis on layout-to-image (L2I) models for simplicity. With the control of layout, L2I models [27, 8, 59] improve compositionality by generating objects at specified locations. For instance, GLIGEN [27] designs trainable gated self-attention layers to incorporate layout input and controls the strength of its incorporation by changing parameter β. Although L2I models improve the weaknesses of compositional text-to-image generation, their generated images exhibit a significant decline in realism compared to T2I models [27, 78] .Prompt: On a wooden table, there is a white vase with a pink flower inserted in it, and a cat is sitting on the table to the right of the vase. We conducted experiments to analyze why a significant decrease in image realism exists. We analyze the layout injection mechanism in GLIGEN [27] by controlling the density of layout through parameter β. As shown in Fig. 1 (a) and (c), our experiments indicate that the density of layout directly influences the realism of generated images. As the control of layout gradually increases, the generated images become less aesthetic and more unstable. This demonstrates that layout and text, as different control conditions, guide the model towards different generation directions, with the former emphasizing compositionality and the latter emphasizing realism. To alleviate this issue, some models [28, 27] leverage the early-stage localization capability of diffusion models [71, 49] and incorporate layouts only during the initial denoising phase. In the later denoising stage, only use text to balance image realism. However, we found this approach yielded minimal effectiveness. We assumed β = 1 in the first t denoising steps and β = 0 in the subsequent denoising steps. As shown in Fig. 1 (b), the object's position is already determined around 20 steps. However, it is common that the generated images exhibit almost no difference between t = 20 and t = 50. This suggests that even when the injection of layout is stopped in the later denoising stages, the control of text alone still fails to alleviate the poor realism issue. The trade-off between realism and compositionality in T2I and L2I models is challenging yet necessary.To this end, we introduce a general training-free and transferred-friendly text-to-image generation framework RealCompo, which utilizes a novel balancer to achieve dynamic equilibrium between realism and compositionality in generated images. We first utilize LLMs to generate scene layouts from text prompt through in-context learning [32] . Then we propose an innovative balancer to dynamically compose pre-trained fidelity-aware (T2I, stylized T2I) and spatial-aware (e.g., layout, keypoint, segmentation map) image diffusion models. This balancer automatically adjusts the coefficient of the predicted noise for each model by analyzing their cross-attention maps during the denoising stage. By combining the respective strengths of the two models, it achieves a trade-off between realism and compositionality. Finally, we extend RealCompo to various spatial-aware conditions through a general compositional denoising process. Moreover, by changing the T2I model to a stylized T2I model, Realcompo can seamlessly achieve compositional generation specified with a particular style. These dramatically demonstrate the great generalization ability of RealCompo. Although there exist methods [61, 2] for composing multiple diffusion models, their application lacks flexibility because they require additional training and cannot be generalized to other conditionss and models. Our method effectively composes two models in a training-free manner, allowing for a seamless transition between various models.To the best of our knowledge, RealCompo effectively achieves a trade-off between realism and compositionality in text-to-image generation. Choosing one (stylized) T2I model and one spatialaware (e.g., layout, keypoint, segmentation map) image diffusion model, RealCompo automatically balances their fidelity and spatial-awareness to realize a collaborative generation. We expands the family of model ensembling/checkpoint merging techniques, which are extensively used in the diffusion community. We believe RealCompo opens up a new research perspective in controllable and compositional image generation.Our main contributions are summarized as the following:• We introduce a new training-free and transferred-friendly text-to-image generation framework RealCompo, which enhances compositional text-to-image generation by balancing the realism and compositionality of generated images.• We design a novel balancer to dynamically combine the predict noise from T2I model and spatial-aware (e.g., layout, keypoint, segmentation map) image diffusion model.• RealCompo has strong flexibility, can be generalized to balance various (stylized) T2I models and spatial-aware image diffusion models and can achieve high-quality compositional stylized generation. It provides a fresh perspective for compositional image generation.• Extensive qualitative and quantitative comparisons with previous outstanding methods demonstrate that RealCompo has significantly improved the performance in generating multiple objects and complex relationships. 
related work: 
 Text-to-Image Generation In recent years, the field of text-to-image generation has made remarkable progress [47, 60, 36, 18, 11, 74, 63] , largely attributed to breakthroughs in diffusion models. By training on large-scale image-text paired datasets, T2I models such as Stable Diffusion (SD) [41] ,DALL-E 2/3 [39, 4] , MDM [17] , and Pixart-α [7] , have demonstrated remarkable generative capabilities. However, there is still significant room for improvement in compositional generation when text prompts include multiple objects and complex relationships [58] . Many studies have attempted to address this issue through controllable generation [72] by providing additional conditions such as segmentation map [22] , scene graph [62] , layout [77] , etc., to constrain the model's generative direction to ensure the accuracy of the number and position of objects in the generated images. However, due to the constraints of the additional conditions, image realism may decrease [27] . Furthermore, several works [37, 9, 68, 65, 30] have attempted to bridge the language understanding gap in models by pre-processing prompts with Large Language Models (LLMs) [1, 48] . It is challenging for T2I models to achieve trade-off between realism and compositionality [65] of generated images.Compositional Text-to-Image Generation Recently, numerous methods have been introduced to improve compositional text-to-image generation [53, 78, 69, 55, 25, 29] . These methods enhance diffusion models in attribute binding, object relationship, numeracy, and complex prompts. Recent studies can generally be divided into two types [52] : one primarily uses cross-attention maps for compositional generation [31, 24, 76] , while the other provides more conditions (e.g., layout, keypoint, segmentation map) to achieve controllable generation [16, 78] . The first methods delve into a detailed analysis of cross-attention maps, particularly emphasizing their correspondence with the text prompt. Attend-and-Excite [6] dynamically intervenes in the generation process to improve the model's generation results in terms of attribute binding (such as color). Most of the second methods offer layout as a constraint, enabling the model to generate images that meet this condition. This approach directly defines the area where objects are located, making it more straightforward and observable compared to the first type of methods [27] . LMD [28] provides an additional layout as input with LLMs. Afterward, a controller is designed to predict the masked latent for each object's bounding box and combine them in the denoising process. However, these algorithms are unsatisfactory in the realism of generated images. A recent powerful framework RPG [65] utilizes Multimodal LLMs to decompose complex generation tasks into simpler subtasks to obtain satisfactory realism and compositionality of generated images. Orthogonal to this work, we achieve dynamic equilibrium between realism and compositionality by combining T2I and spatial-aware image diffusion models. 
method: 
 In this section, we introduce our method, RealCompo, which designs a novel balancer to achieve dynamic equilibrium between realism and compositionality of generated images. We initially focus on the layout-to-image models. In Section 3.1, we analyze the necessity of incorporating influence for the predictive noise of each model and provide a method for calculating coefficients. In Section 3.2, we provide a detailed explanation of the update rules employed by the balancer, which utilizes a training-free approach to update coefficients dynamically. In Section 3.3, we provide a universal formula and denoising procedure that enable the balance of T2I models with any spatial-aware image diffusion model, such as keypoint or segmentation-to-image models based on ControlNet [72] . We also extend RealCompo to stylized compositional generation by stylized T2I models. LLM-based Layout Generation. Since spatial-aware conditions are similar essentially, we first choose layout as the representative of spatial-aware condition for introduction. As shown in Fig. 2 , we leverage the powerful in-context learning [57, 79] capability of Large Language Models (LLMs) to analyze the input text prompt and generate an accurate layout to achieve "pre-binding" between objects and attributes. The layout is then used as input for the L2I model. In this paper, we choose GPT-4 for layout generation. Please refer to Appendix C.1 for detailed explanation.Combination of Two Types of Noise. In diffusion models, the model's predicted noise ϵ t directly affects the direction of the generated images. In T2I models, ϵ text t exhibits more directive toward realism [41] , whereas in L2I models, ϵ layout t demonstrates more directive toward compositionality [27] . To achieve the trade-off between realism and compositionality, a feasible but untapped solution is to compose the predicted noise of two models. However, the predicted noise from different models has its own generative direction, contributing differently to the generated results at different timesteps and positions. Based on this, we design a novel balancer that achieves dynamic equilibrium between the two models' strengths at every position i in the noise for timestep t. This is achieved by analyzing the influence of each model's predicted noise. Specifically, we first set the same coefficient for the predicted noise of each model to represent their influence before the first denoising step:EQUATIONIn order to regularize the influence of each model, we perform a softmax operation on the coefficients to get the final coefficients:EQUATIONwhere c ∈ {text, layout}.The balanced noise can be derived according to the coefficient of each model:EQUATION)where ⊙ denotes pixel-wise multiplication.Once the predicted noise ϵ c t and the coefficient Coe c t of each model are provided, the balanced noise can be derived from Eq. 2 and Eq. 3. At timestep t, the balancer dynamically updates coefficients as described in Section 3.2.The alignment between the generated images and the input prompts is largely influenced by model's cross-attention maps, which encapsulate a wealth of matching information between visual and textual elements, such as location and shape. Specifically, given the intermediate feature φ(z t ) and the text embeddings τ θ (y), cross-attention maps can be derived in the following manner:EQUATIONEQUATIONwhere Q and K are respectively the dot product results of the intermediate feature φ(z t ), text embeddings τ θ (y), and two learnable matrices W Q and W K . A ij defines the weight of the value of the j-th token on the i-th pixel. Here, j ∈ {1, 2, . . . , N (τ θ (y))}, and N (τ θ (y)) denotes the number of tokens in τ θ (y). The dimension of K is represented by d k .Update Rule of Dynamic Balancer. We designed a novel balancer that dynamically balances two models according to their cross-attention maps at timestep t. Specifically, we represent layout asB = {b 1 , b 2 , . . . , b v }, which is composed of v bounding boxes b.Each bounding box b corresponds to a binary mask M b , where the value inside the box is 1 and the value outside the box is 0. Given the predicted noise ϵ c t and the coefficient Coe c t of each model, the balanced noise ϵ t and denoised latent z t-1 can be derived from Eq. 3 and Eq. 12. By feeding z t-1 into two models, we obtain the cross-attention maps A c t-1 output by the two models at timestep t -1, which indicates the denoising quality feedback after the noise ϵ c t of the model at time t is weighted by ξ c t . Based on A c t-1 , we define the loss function as follows:EQUATIONwhere c ∈ {text, layout}, j b denotes the token corresponding to the object in bounding box b. Since two models are controlled by different conditions, averaging the predicted noise equally will lead to instability in the generated images. This is because the T2I model breaks the layout constraints of the L2I model, reducing the compositionality of the generated images, as we have demonstrated in experimrnts in Fig. 9 . Therefore, we designed this loss function to measure the alignment between the cross-attention maps and layout for each model. A smaller loss indicates better compositionality.The following rule is used to update Coe c t :EQUATIONwhere ρ t is the updating rate. This update rule continuously strengthens the constraints on both models by assessing the positional alignment of the layout within the cross-attention maps, ensuring the maintenance of the localization capability of L2I model while injecting fidelity information of T2I model. It is worth noting that previous methods [6, 59, 28] for parameter updates based on function gradients were primarily using energy functions to update latent z t . We are the first to update the influence of predicted noise based on the gradient of the loss function, which is a novel and stable method well-suited to our task. The complete denoising process is detailed in Appendix C.3.Other spatial-aware text-to-image diffusion models are essentially similar to L2I models. Keypointto-image (K2I) models generate specified actions or poses within each group of keypoints region, and segmentation-to-image (S2I) models fill indicated objects within each segmented region. The concept of "region" is always present, which transforms T2I generation from a macro perspective to utilizing region-based control for T2I generation from a micro perspective. This concept is also the core of enhancing image compositionality. Compared with layout-based T2I generation, the only difference is that keypoints and segmentation maps have stronger control over the model based on regions, requiring that the pose is maintained and the object is correct and unique.A group of four friends, two girls and two boys wearing casual clothing, laughing together as they walk along a flower-lined path beneath a cherry blossom tree in full bloom.A girl with an angelic appearance dances in a white dress as dark clouds gather in the sky, creating a somber atmosphere.In the living room there are two white sofas and a white coffee table, and the living room has two oversized floor-to-ceiling windows.A house with dark red walls surrounded by flowers and trees. General Form for Extension to Other Spatial-Aware Conditions We rethink Eq. 6, which is RealCompo's core approach in combining T2I and L2I models, where the only layout-related variable is the binary masks M. Considering that spatial-aware controllable T2I generation inherently focus on the concept of "region control", we introduce a transfer function:EQUATIONwhere C represents other spatial-aware conditions such as keypoint and segmentation map. f (•) represents the calculation of the minimum and maximum values of the horizontal and vertical coordinates occupied by each set of keypoints or a segmentation block within the entire image coordinate system, which can be transformed into a layout and a binary mask M. Therefore, for any T2I models with spatial-aware control, the general loss function of RealCompo is:EQUATIONwhere c ∈ {text, spatial}. Similarly, Coe c t is dynamically updated using Eq. 7. ControlNet [72] enables controllable T2I generation based on various spatial-aware conditions. In this work, the spatial-aware branches besides layout are all based on ControlNet, which is illustrated in Fig. 4 . The generated images of keypoint-and segmentation-based RealCompo are shown in Fig. 3 .As an essential indicator of fidelity, image style [50, 67] guides us to expand the application potential of RealCompo. Since RealCompo mainly leverages T2I models to enhance and guide the realism and aesthetic quality of generated images. By replacing the T2I model with various stylized T2I models and combining it with a spatial-aware image diffusion model, we can achieve outstanding compositional generation under this style. The experiments are shown in Fig 8 . 
experimental setup: 
 Implementation Details Our RealCompo is a generic, scalable framework that can achieve the complementary advantages of the model with any chosen (stylized) T2I models and spatial-aware image diffusion models. We selected GPT-4 [1] as the layout generator in our experiments, the detailed rules are described in Appendix C.1. For layout-based RealCompo, we chose SD v1.5 [41] and GLIGEN [27] as the backbone. For keypoint-based RealCompo, we chose SDXL [4] and Table 1 : Evaluation results about compositionality on T2I-CompBench [21] . RealCompo consistently demonstrates the best performance regarding attribute binding, object relationships, numeracy and complex compositions. We denote the best score in blue , and the second-best score in green . The baseline data is quoted from PixArt-α [7] . [41] , as well as the layout-to-image models, GLIGEN [27] and LMD+ [28] .Colored text denotes the advantages of RealCompo in generated images.ControlNet [72] as the backbone. For segmentation-based RealCompo, we chose SD v2.1 [41] and ControlNet [72] as the backbone. For style-based RealCompo, we chose two stylized T2I models: Coloring Page Diffusion and CuteYukiMix as the backbone, and chose GLIGEN [27] as the backbone of L2I model. All of our experiments are conducted under 1 NVIDIA 80G-A100 GPU.Baselines and Benchmark To evaluate compositionality, we compare our RealCompo with the outstanding T2I and L2I models on T2I-CompBench [21] . This benchmark test models across aspects of attribute binding, object relationship, numeracy and complexity. To evaluate realism, we randomly select 3K text prompts from the COCO validation set , we utilize ViT-B-32 [10] to calculate the CLIP score and LAION aesthetic predictor to calculate aesthetic score, reflecting the degree of match between generated images and prompts as well as the aesthetic quality, respectively. In addition to objective evaluations, we conducted a user study to evaluate RealCompo and stylized RealCompo in terms of realism, compositionality, and comprehensive evaluation.Results of Compositionality: T2I-CompBench We conducted tests on T2I-CompBench [21] to evaluate the compositionality of RealCompo compared to the outstanding T2I and L2I models. As demonstrated in Table 1 , RealCompo achieved state-of-the-art performance on all seven evaluation tasks. It is clear that RealCompo and L2I models GLIGEN [27] and LMD+ [28] show significant improvements in spatial-aware tasks such as spatial and numeracy. Our method improves image authenticity through conditional control while maintaining detail and aesthetic quality.attributed to the guidance provided by the additional conditions, which greatly enhances the model's compositional performance. RealCompo employs a balancer for better control over positioning, boosting its advantages in these aspects. However, the L2I models exhibit a noticeable decline in performance on tasks like texture and non-spatial. This decline is due to the injection of layout embeddings, which dilute the density of text embeddings, leading to suboptimal semantic understanding by the model. By composing additional T2I models, RealCompo provides sufficient textual information during the denoising process and achieves outstanding results in tasks that reflect realism, such as texture, non-spatial and complex tasks. As shown in Fig. 5 , compared with the current outstanding L2I models GLIGEN and LMD+, RealCompo achieves a high level of realism while keeping the attributes of the objects matched and the number of positions generated correctly. We attribute this to the dynamic balancer, which enhances image realism and aesthetic quality while maintaining high compositionality.User Study In addition to objective evaluations, we designed a user study to subjectively assess the practical performance of various methods. We randomly selected 15 prompts, including 5 for stylization experiments. Comparative tests were conducted using T2I models, spatial-aware image diffusion models, and RealCompo. We invited 39 users from diverse backgrounds to vote on image realism, image compositionality, and comprehensive evaluation, resulting in a total of 1755 votes. As illustrated in Fig. 6 , RealCompo received widespread user approval in terms of realism and compositionality.We provide examples from the user study in Fig. 7 , which demonstrates the advantages of RealCompo over the T2I model in realism. As shown in Fig. 7 (a), T2I model generates a teapot that is visibly suspended in the air, which doesn't conform to the physical laws of real-world scenes. In contrast, RealCompo generates objects within reasonable Results of Extend Applications: More Spatial-Aware Conditions We extend RealCompo to more spatial-aware controlled image generation. As shown in Fig. 3 , keypoint-and segmentationbased RealCompo achieves outstanding performance in both realism and compositionality. This promising result reveals that as spatial-aware conditions, layout, keypoint, and segmentation map are fundamentally similar, RealCompo focuses on these similarities and achieves a general generative paradigm for compositional generation.Results of Extend Applications: Stylized Generation Image style is an essential indicator of fidelity. We experiment with generalizing RealCompo to various pre-trained stylized T2I models. We selected the Coloring Page Diffusion and Cutyukimix as the foundational stylized models, focusing on the coloring page style and adorable style, respectively. As shown in Fig. 8 , RealCompo perfectly inherits the style of the T2I models and, with the help of L2I model, achieves powerful compositional generation under these styles, which is currently difficult for stylized diffusion models to accomplish. We found it difficult for LMD to strictly maintain the style by simply replacing the backbone with a stylized model, often leading to text leakage [13] . For example, terms like "crayon" frequently appear in the coloring page style, indicating that the layout control disrupts the style or text control, making it challenging for L2I models to achieve stylized compositional generation. In contrast, by maintaining image realism and style, RealCompo demonstrates strong compositionality while better preserving the style compared to currently outstanding stylized models like InstantStyle [50] .Importance of Dynamic Balancer As shown in Fig. 9 , we conducted experiments on the importance of the dynamic balancer. It is clear that without the use of the dynamic balancer, the generated images do not align with the layout. This is because the predicted noise in T2I model is not constrained by the layout, leading to the model generating the object at any position, and the quantity is uncontrollable. Although the image realism is high, the predicted noise of T2I model disrupts the object distribution of the predicted noise of L2I model, leading to poor compositionality of the generated images and uncontrollable in the generation process.Generalizing to Different Backbones To explore the generalizability of RealCompo for various models, we choose two T2I models, SD v1.5 [41] and TokenCompose [54] , and two L2I models, GLIGEN [27] and LayGuide (Layout Guidance) [8] . We combine them two by two, yielding four TokenCompose+LayGuide Two cute small corgi sitting in a movie theater with two popcorns in front of them.A cruise ship at sea and a lighthouse on the right.Two sunflowers are growing in a blue flowerpot, and a butterfly dances gracefully. versions of RealCompo v1-v4. The experimental results are shown in Fig. 9 . The four versions of RealCompo all have a high degree of realism in generating images and achieving desirable results regarding instance composition. This is attributed to the dynamic balancer combining the strengths of T2I and L2I models, and it can seamlessly switch between models because it is simple and requires no training. We also found that RealCompo, when using GLIGEN as the L2I model, performs better than when using LayGuide in generating objects that match the layout. For instance, in the images generated by RealCompo v4 in the first and third rows, "popcorns" and "sunflowers" do not fill up the bounding box, which can be attributed to the superior performance of the base model GLIGEN compared to LayGuide. Therefore, when combined with more powerful T2I and L2I models, RealCompo is expected to yield more satisfactory results. 
conclusion: 
 In this paper, to solve the challenge of complex or compositional text-to-image generation, we propose the SOTA training-free and transferred-friendly framework RealCompo. In RealCompo, we propose a novel balancer that dynamically combines the advantages of various (stylized) T2I and spatial-aware (e.g., layout, keypoint, segmentation map) image diffusion models to achieve the trade-off between realism and compositionality in generated images. In future work, we will continue to improve this framework by using a more powerful backbone and extend it to more realistic applications.supplementary material is structured into several sections that provide additional details and analysis related to our work on RealCompo. Specifically, it will cover the following topics:• In Appendix A, we provide a preliminary about Stable Diffusion.• In Appendix B, we rovide more visualized results to verify the generality of the phenomenon we discovered in our motivation. • In Appendix C.1, we provide a detailed pipeline about how to get layout through in-context learning of LLMs. • In Appendix C.2, we provide a detailed proof of the existence of the gradient in Eq. 7.• In Appendix C.3, we provide the pseudocode for RealCompo to thoroughly demonstrate its denoising process. • In Appendix C.4, we conduct a detailed analysis of the gradient changes of the two models in Eq. 7 during the denoising process. • In Appendix C.5, we analysis the limitations and future work of RealCompo.• In Appendix C.6, we analysis the broader impact of RealCompo.• In Appendix D, we provide more additional visualized results. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We have also give detailed instructions about experiment setup in Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA]Justification: Error bars are not reported because it would be too computationally expensive.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We disscuss the limitications of the work in Appendix C.6Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The creators or original owners of code used in the paper are properly credited, and the license and terms of use are explicitly mentioned and properly respected.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: New assets introduced in the paper are well documented. We provide them as supplementary material.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used.  
 
 Appendix: Diffusion models [19, 44, 5] are probabilistic generative models. They can perform multi-step denoising on random noise x T ∼ N (0, I) to generate clean images through training. Specifically, a gaussian noise ϵ is gradually added to the clean image x 0 in the forward process:EQUATIONwhere ϵ ∼ N (0, I) and α t is the noise schedule.Training is performed by minimizing the squared error loss:EQUATIONThe parameters of the estimated noise ϵ θ are updated step by step by calculating the loss between the real noise ϵ and the estimated noise ϵ θ (x t , t).The reverse process aims to start from the noise x T , and denoise it according to the predicted noise ϵ θ (x t , t) at each step. DDIM [45] is a deterministic sampler with denoising steps:EQUATIONStable Diffusion [41] is a significant advancement in this field, which conducts noise addition and removal in the latent space. Specifically, SD uses a pre-trained autoencoder that consists of an encoder E and a decoder D. Given an image x, the encoder E maps x to the latent space, and the decoder D can reconstruct this image, i.e., z = E(x), x = D(z). Moreover, Stable Diffusion supports an additional text prompt y for conditional generation. y is transformed into text embeddings τ θ (y) through the pre-trained CLIP [38] text encoder. ϵ θ is trained via:EQUATIONIn the inference process, noise z T ∼ N (0, I) is sampled from the latent space. By applying Eq. 12, we perform step-by-step denoising to obtain a clean latent z 0 . The generative image is then reconstructed through the decoder D.A cute rabbit is sitting on a wooden table, and there is a teddy bear and a vase on its right side. To further verify the generality of the phenomenon we discovered in our motivation. As shown in Fig. 10 , we first conducted more experiments on GLIGEN [27] . We observed that as the layout control increased (with a higher β) or the number of layout control steps increased (with a higher t 0 ), the realism of the generated images declined. There is a noticeable degradation in both detail richness and aesthetic quality. For instance, the legs of the teddy bear appear unrealistic, as if it is facing backward with strange distortions, and the overall details of the rabbit become blurred and unappealing.Similarly, as shown in Fig. 11 , we performed experiments using InstanceDiffusion [53] , where we also define a parameter β to control the strength of the layout control. It is evident that there is significant quality degradation in the dog's facial and body details. Additionally, the cat's eyes are different sizes, and the bird's legs are abnormally thin, indicating reduced realism in the generated images under the influence of layout control. This suggests that achieving a balance between realism and compositionality in generated images is generally unattainable.Large Language Models (LLMs) have witnessed remarkable advancements in recent years [48, 23, 51, 75, 70] . Due to their robust language comprehension, induction, reasoning, and summarization capabilities, LLMs have made significant strides in the Natural Language Processing (NLP) tasks [15, 56] . In the context of multiple-object compositional generation, text-to-image diffusion models a relatively weaker understanding of language, as reflected in the poor compositionality of the generated images. Consequently, exploring ways to harness the inferential and imaginative capacities of LLMs to facilitate their collaboration with text-to-image diffusion models, thereby producing images that adhere to the prompt, offers substantial research potential.In our task, we leverage LLMs to directly infer the layout of all objects based on the user's input prompt through in-context learning (ICL) [26, 43] . This layout is used for the layout-to-image model of RealCompo, eliminating the need to manually provide a layout for each prompt and achieve pre-binding of multiple objects and attributes. Specifically, as shown in Fig. 12 , we construct prompt templates, which include descriptions of task rules (instruction), in-context examples (demonstration), and the user's input prompt (test). Through imitation reasoning based on the instruction, LLM generate layout for each object, where each layout represents the coordinates of the top-left and bottom-right corners of a respective box. We selected the highly capable GPT-4 [1] as layout generator. C.2 Analysis of the Existence of in Eq. 7Here we set:EQUATIONIf the loss function is given by Eq. 6, the gradient in Eq. 7 can be derived as follows:EQUATION∂A c(j b ,t-1)∂z t-1 ∂z t-1 ∂ϵ t ∂ϵ t ∂ξ c t ∂ξ c t ∂Coe c t   = b    ∂L b A text t-1 , layout t-1 ∂A c (j b ,t-1)∂A c(j b ,t-1)EQUATION∂A c(j b ,t-1)EQUATION∂A c(j b ,t-1)EQUATIONFor any T2I and L2I models, we have the following:EQUATION)where J is a matrix with all elements equal to 1. All variables in Eq. 15 are known, indicating the existence of the gradient in Eq. 7.using the loss function given by Eq. 9 under any spatial-aware conditions, the gradient in Eq. 7 can be derived as follows:∂L A text t-1 , A spatial t-1 ∂Coe c t = b   ∂L b A text t-1 , A spatial t-1 ∂A c (j b ,t-1)∂A c(j b ,t-1)∂z t-1 ∂z t-1 ∂ϵ t ∂ϵ t ∂ξ c t ∂ξ c t ∂Coe c t   = b   ∂L b A text t-1 , A spatial t-1 ∂A c (j b ,t-1)∂A c(j b ,t-1)EQUATIONEQUATIONwhere c ∈ {text, spatial}.Therefore, the gradient in Eq. 7 exists for the selection of different loss functions.We provide a detailed compositional denoising process for RealCompo, which achieves a complementary balance between the advantages of the T2I model and the spatial-aware diffusion model by combining their predicted noise during the denoising stage. We provide the pseudocode for the compositional denoising process of the layout-based RealCompo as followed, we have highlighted the innovations of our method in blue. if t > t 0 then 5: ϵ t , _ = L2I (z t , P, B, t) 6:EQUATIONEQUATIONGet the balanced noise ϵ t from Eq. 2 and Eq. 3 10:Get the denoised latent z t-1 from Eq. 12 11:ϵ text t-1 , A text t-1 = T2I (z t-1 , P, t)12:EQUATIONCompute L(A text t-1 , A layout t-1 ) from Eq. 614:Update Coe c t according to Eq. 715:Get the balanced noise ϵ t from Eq. 2 and Eq. 3 Get the denoised latent z t-1 from Eq. 12 18: end for 19: return z 0Gradient Analysis We RealCompo v3 and v4 to analyze the gradient changes in Eq. 7 across all denoising stages. As shown in Fig. 13 , we use the same prompt and random seed to visualize the gradient magnitude changes corresponding to T2I and L2I for each model version. We observe that the gradient magnitude change of RealCompo v4 fluctuated more in the early denoising stages. We argue that TokenCompose, which enhances the composition capability of multiple-object generation by fine-tuning the model using segmentation masks, may overlap in functionality with the layout-based multiple-object generation, and TokenCompose's positioning of objects may not consistently align with the bounding box. Therefore, RealCompo must focus on balancing the positioning of TokenCompose and layout in the early denoising stages, leading to less stable gradients compared to RealCompo v3. Additionally, due to LayGuide's weaker positioning ability compared to GLIGEN, RealCompo v4 may occasionally generate objects with less coverage of the bounding box, as mentioned in the ablation experiment in Section 4.3. Limitations While our RealCompo enhances both realism and compositionality in a training-free manner, it should be noted that the computational cost of our method is slightly higher compared to that of a single T2I model or a single spatial-aware image diffusion model, due to the need to combine two models and compute loss and gradients. However, by adjusting the combination stage of RealCompo, we can keep the computational cost within an acceptable range.In future work, we aim to explore more efficient computational methods to improve the calculation efficiency of RealCompo while maintaining high-quality results and we plan to extend its application to more challenging tasks such as text-to-video and text-to-3D generation. Furthermore, given that the exceptional classifier-free guidance strategy employs fixed weights, we aim to explore the potential of using fixed coefficients to further enhance the capabilities of RelCompo.Recent significant advancements in text-to-image diffusion models have opened up new possibilities for creative design, autonomous media, and various other sectors. However, the dual-use nature of this technology raises concerns about its social impact. Image diffusion models carry the risk of misuse, particularly in the realm of impersonating humans. For example, in today's society, malicious applications such as "deepfakes" have been employed in inappropriate contexts to fabricate attacks on specific public figures. It is crucial to clarify that our algorithm is designed to enhance the quality of image generation, and we do not endorse or facilitate such malicious applications.A group of people are watching a fireworks display.A blue bird standing on a branch with two yellow blossoms on the right.A child is reading a book under a tree.A person is holding a camera and taking photos of the city.Three cats are sitting on the table, with a plate of food in front of them.A brown teddy bear holding a blue water cup sits on the sofa. Five men stand together in a line, serious in expression.An airplane parked on the runway as the sun sets behind it.Two cats sitting on the windowsill looking at each other. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 


Decision: All the reviewers recommend acceptance of the work. Reviewers appreciated the simple yet effective technique with good results. Reviewers raised several concerns related to novelty and clarification questions, several of which are addressed in the rebuttal. Reviewers did raise valid concerns and authors are encouraged to do the best of their abilities to include the rebuttal discussions and address all the concerns in the final version.

Reviews: review: paper_summary: paper_summary: The paper introduces a training-free and flexible text-to-image generation framework called RealCompo, which enhances compositional text-to-image generation by balancing the realism and compositionality of generated images. It features a novel balancer that dynamically combines the predicted noise from T2I models and spatial-aware image diffusion models (such as layout, keypoint, segmentation map). This framework provides a fresh perspective for compositional image generation.
strength_weakness: strength_weakness: 1. The biggest problem is novelty/contribution. The paper uses a balancer to balance two diffusion models, which is very similar to the concept of "checkpoint merge" / "model ensemble". It is necessary to clarify the differences between these two approaches.
questions: questions: I will consider increasing the score or maintaining it if the authors can address the following issues.
1. The paper uses a balancer to balance two diffusion models, which is very similar to the concept of "checkpoint merge" or "model ensemble". It is necessary to clarify the differences between these two approaches.
2. Please explain the necessity of the proposed method. If condition-guided models like ControlNet use a stronger backbone, higher quality, larger quantity, and more diverse data and annotations during training, is it still necessary to balance with non-condition-guided text-to-image models?
limitations: limitations: Yes.
paper_summary: paper_summary: This paper presents a method, named RealCompo, for combining multiple diffusion models, such as a text-to-image model and a spatial-aware one, to achieve the best of both worlds: superior image realism and compositionality. It merges predicted noise from both models during each denoising step, and balances them based on two sets of coefficients that are dynamically updated through a loss that forces cross attention maps to follow provided spatial constraints. Experiments show that RealCompo improves both realism and compositionality compared to using either model alone, and it outperforms SOTA spatial-aware models in compositional generation.
strength_weakness: strength_weakness: - Since the proposed method runs multiple models concurrently and requires gradient-based updates, a discussion on computational efficiency is crucial for its practical application.
- Figure 1 does not illustrate the trade-off between realism and compositionality well. I don't see an apparent drop of realism/aesthetics as $\beta$ (control of layout) increases.
questions: questions: - Why can RealCompo improve image realism (Table 2, Figure 6 left) and style preservation (Figure 6 right) beyond the capabilities of the T2I model alone? This is difficult to understand because the loss function (Eq. 6) employed for balancing the two models is designed solely to encourage better adherence to the provided spatial constraints. I will reconsider my rating if this aspect can be properly addressed.
- What is the purpose of initializing the coefficients as random values (Eq. 1)? This seems unnecessary and introduces random bias towards pixels.
limitations: limitations: The authors touched on the computational cost in Appendix B.5. However, a more detailed comparison with T2I and L2I models concerning computational overhead and memory overhead is crucial to determine the feasibility of the proposed method.
paper_summary: paper_summary: This paper proposes to balance the realism and compositionality of the generated images by means of different diffusion models, such as pre-trained T2I models and models based on spatial perceptual control. The authors develop a balancer that optimizes the two coefficients through mask-based attention manipulation to enable dynamic combination of the different models. They also demonstrate the effectiveness of the method through extensive experiments.
strength_weakness: strength_weakness: Cons:
1. Unclear Task definition. First, even different pre-trained T2I models can result in different so-called REALISM due to different training strategies and data. Secondly, Control-based models, e.g., Controlnet and GLIGEN, all add controllability while inheriting the realism generated by the original T2I models. So I'm not sure why the authors need to propose another method to do the so-called balancing of realism and composability.In addition, how can you guarantee that the combination of T2I and L2I can generate real diagrams when the images generated by T2I are all more fake?
2. Unconvincing observation. In Figure 1 and paragraphs 2 to 4, the authors present their motivation only through an existing method, GLIGEN. I argue that this is inappropriate. After all, there are many methods based on layout or control, no matter what conditions these methods use, such as layout, segmentation map, and pose. Motivation is the key to a paper and cannot be deduced from just one method. So I argue that this observation is not very convincing to me.
3. Insufficient contribution. It is common to use optimization to produce better images, but the difference lies in the design of the optimization. The use of mask to combine with attention to do optimization is more common way, such as in [1,2,3,4,5]. The only difference is that the existing methods may optimize zt, this paper optimizes two coefficients. So I argue that the contribution of this paper is not very sufficient.
questions: questions: 1. Fig-1 lacks a detailed explanation.
2. Sub-sec 3.3 contains not much valuable information, as the only difference between Eq.9 and Eq.6 is how to obtain a mask/spatial condition. 

I will revise my rating according to the author's feedback and the reviewer's discussion.
limitations: limitations: N/A

Comments: title: Thank you for your support
Thank you for your support
The rebuttal addressed my concerns.
Gentle Reminder
Thank you for your support
Response to Reviewer XVVX
please plot the coefficent curve
Response to Reviewer juPc
A minor clarify
A minor clarify
Response to Reviewer juPc
comment: Thank you very much for raising the score! We sincerely appreciate your valuable comments and the time and effort you put into reviewing our paper. 

We will make sure to incorporate these suggestions into the final manuscript.

Warm Regards,

The Authors
My questions are all well addressed, and I decided to increase my rating to weak accept. BTW, PLEASE make the necessary revisions in the final version according to the questions listed above.
Thank you very much for raising the score! We sincerely appreciate your valuable comments and the time and effort you put into reviewing our paper. 

Based on your thoughtful suggestion of a fixed-coefficient strategy, we make sure to plan additional experiments to analyze its feasibility and identify the optimal fixed coefficient to enhance the efficiency and quality of our method.

Thank you once again for your invaluable contribution to our research.

Warm Regards, 

The Authors
1. The rebuttal has satisfactorily addressed my concerns regarding real-world applications, particularly the issue that the proposed method requires gradient backpropagation, which demands more memory and increases inference time. The training-free approach introduced in this paper achieves remarkable improvements in both controllability and realism. It also expands the family of model ensembling/checkpoint merging techniques, which are extensively used in the diffusion community. Considering that classifier-free guidance also requires double the forward passes but results in exceptional performance, I am inclined to view this paper positively.
2. Given the potential of this paper to apply a fixed coefficient that eliminates the need for gradient backpropagation, I am raising my score from weak accept to accept. Nevertheless, I strongly recommend that the authors provide further analysis on the use of a fixed coefficient. For example, in real-world applications, although we can apply an optimal or dynamic CFG for different models, prompts, or timesteps, a fixed constant CFG strategy is often employed and still results in significant improvements. Additionally, a fixed dynamic CFG for different timesteps that does not require gradient backpropagation, such as cosine CFG or the CFG used by MDT, could be considered.
3. Leveraging large language models (LLMs) to orchestrate various tools and utilize multiple models to complete complex tasks—such as generating layouts, keypoints, or segmentation as intermediate steps—represents a promising future direction. This approach could provide more effective solutions for challenging problems like multi-object generation and attribute binding.
Hello, Reviewer ZHfz, Regarding your concerns, we have provided some responses that you might find useful, including:

- We have highlighted and summarized the innovations and contributions of our method from two perspectives: innovation in task establishment and model design.

- Additional experimental results, incuding using different models to validate our motivation and compare our method with T2I models in terms of the visual authenticity of the images.

- We provide a detailed explanation of Figure 1 in the paper and the motivation of our method.

We sincerely hope our responses address some of your concerns. If you have any further questions, please feel free to ask. Thank you.
Thank you very much for raising the score! We sincerely appreciate your valuable comments and the time and effort you put into reviewing our paper. We will make sure to incorporate these suggestions into the final manuscript.

Warm Regards, 

The Authors
Thank you for the response! In the following we provide additional explanations regarding to your questions. Please feel free to let us know if these address your concerns.

**Q1: It's recommended to plot the current dynamic coefficient curve.**

A1: Thank you for your suggestion. Unfortunately, **due to NeurIPS's requirement that no anonymous links be included in the rebuttal**, we regret that we cannot provide you with the dynamic coefficient curve at this time. However, we will certainly include this experiment and its analysis in the revised manuscript. Just a gentle reminder, **in Figure 10 of our paper**, we present the curve showing the gradient of the loss function (Eq 6) with respect to the coefficient during the denoising process. The figure illustrates that the coefficient exhibits significant convergence during denoising, and there are distinct update patterns for the coefficient when using different backbones in RealCompo.

**Q2: Is there exisits a general coefficient curve that is suitable for most of the prompt? If it is possible, there's no need for an optimization-based method that requires gradient backward, which will be mush easier for real application.**

A2: It is a valuable view to explore a general coefficient curve. However, we observed that the coefficient varies depending on the prompt based on extensive experiments. This is because **the coefficient is primarily optimized based on the layout, and different prompts correspond to different layouts, leading to varying coefficients**. We will certainly include visualizations and analysis of this part in the revised manuscript. Additionally, as seen in Figure 10, **RealCompo with different backbones also requires distinct coefficient update strategies**.

But I agree that having a general coefficient curve would make the application of our method more convenient and meaningful. Thank you for your thoughtful suggestions. This is a preliminary attempt, and we will explore the potential of a general coefficient curve in the future.

**Q3: It's recommended to show the results of the fixed coefficient (the best selected) instead of dynamic coefficient.**

A3: Thank you for your suggestion. **In the third column of Figure 8** (w/o Dynamic Balancer) in the paper, we present the results of experiments **using a simple fixed coefficient, where both models have the same coefficient**. The figure illustrates that without dynamically updating the coefficient, the T2I model, which lacks layout constraints, negatively impacts the positioning capability of the L2I model. This results in generated images where the object positions do not align with the layout, despite the T2I model retaining a higher realism advantage. We will include additional manually designed fixed coefficients in the manuscript to validate the effectiveness of our method.
I would like to thank the authors for their detailed explanations. Having considered the responses to my questions and those of other reviewers, I am changing my rating from borderline reject to weak accept.
1. It's recommended to plot the current dynamic coefficient curve.
2. Is there exisits a general coefficient curve that is suitable for most of the prompt? If it is possible, there's no need for an optimization-based method that requires gradient backward, which will be mush easier for real application.
3. It's recommended to show the results of the fixed coefficient (the best selected) instead of dynamic coefficient.
Thank you for the response! In the following we provide additional explanations regarding to your questions. Please feel free to let us know if these address your concerns.

**Q1: The improved aesthetic quality, as indicated by Table 2 and the second paragraph of Sec 4.2, is still hard to comprehend.**

A1: We would like to emphasize that the aesthetic predictor was trained using the Aesthetic Visual Analysis (AVA) and Simulacra Aesthetic Captions (SAC) datasets, **both of which are annotated by experts or photography enthusiasts**. In these datasets, each image is given a score ranging from 1 to 10 based on its perceived appeal. **It is important to note that these annotators tend to be more critical of images that appear unnatural or fake (i.e., images with unrealistic or awkward compositions)**. The aesthetic predictor is designed to filter out high-quality images that excel in both detail representation and composition. The aesthetic scores for the images in Figure 5 in the newly provided PDF are provided as follows:

|                    | Aesthetic Score$\uparrow$ |
| ------------------ | ----------------------------- |
| Fig5 (a)-T2I Model | 5.304                         |
| Fig5 (a)-RealCompo | **5.653**                     |
| Fig5 (b)-T2I Model | 5.269                         |
| Fig5 (b)-RealCompo | **5.775**                     |
| Fig5 (c)-T2I Model | 5.471                         |
| Fig5 (b)-RealCompo | **5.884**                     |

From the table, it can be observed that when the images have similar levels of detail and visual refinement, **the aesthetic predictor assigns a higher aesthetic score to images with more reasonable positioning**.

**Q2: The authors also noticed that "T2I model outperforms in detail and aesthetics" in Fig 5 in the newly provided PDF.**

A2: Thank you for your comment, and we apologize for any confusion caused. Here, we intended to convey that T2I models excel in terms of detail and **visual refinement**. We realize that our use of the term "aesthetic" may have led to confusion, as **it was not meant to refer to the aesthetic score in this context**. We apologize again for any misunderstanding this may have caused.

We will review and refine the manuscript to to clarify the two issues you have raised.
Thank the authors for the clarification. 

I would like to believe the explanation about realism improvement. However, the improved aesthetic quality, as indicated by Table 2 and the second paragraph of Sec 4.2, is still hard to comprehend. Meanwhile, the authors also noticed that "T2I model outperforms in detail and aesthetics" in Fig 5 in the newly provided PDF. I would like the authors to further clarify this.
Dear Esteemed Reviewer ZHfz,

We would like to clarify that the supplement PDF refers to **the pdf in global response**. We apologize for any confusion this may have caused.

Should there be any further points that require clarification or improvement, please know that we are fully committed to addressing them promptly.

Thank you once again for your invaluable contribution to our research.

Warm Regards,

The Authors
Dear Esteemed Reviewer XVVX,

We would like to clarify that the supplement PDF refers to **the pdf in global response**. We apologize for any confusion this may have caused.

Should there be any further points that require clarification or improvement, please know that we are fully committed to addressing them promptly.

Thank you once again for your invaluable contribution to our research.

Warm Regards, 

The Authors
Dear Esteemed Reviewer juPc,

Thank you for your kindly response. We would like to clarify that the supplement PDF refers to **the pdf in global response**. We apologize for any confusion this may have caused.

Should there be any further points that require clarification or improvement, please know that we are fully committed to addressing them promptly.

Thank you once again for your invaluable contribution to our research.

Warm Regards, The Authors
Thank the authors for the detailed response. I appreciate the additional information provided. However, I can't find the mentioned supplementary PDF that contains Figures 3, 4, and 5. I've checked the paper's appendix and the supplementary .zip file (which only has code). Could you please provide this PDF or clarify where to find it?