Product2IMG: Prompt-Free E-commerce Product Background Generation with Diffusion Model and Self-Improved LMM

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 16:30:32

Abstract: In e-commerce platforms, visual content plays a pivotal role in capturing and retaining audience attention. A high-quality and aesthetically designed product background image can quickly grab consumers' attention, and increase their confidence in taking actions, such as making a purchase. Recently, diffusion models have achieved profound advancements, rendering product background generation a promising avenue for exploration. However, text-guided diffusion models require meticulously crafted prompts. The diverse range of products makes it challenging to compose prompts that result in visually appealing and semantically appropriate background scenes. Current work has made great efforts on creating prompts through expert-crafted rules or specialized fine-tuning of large language models, but it still relies on detailed human inputs and often falls short in generating desirable results by e-commerce standards. 

In this paper, we propose Product2Img, a novel prompt-free diffusion model with automatic training data refinement strategy for product background generation. Product2Img employs Contrastive Background Alignment (CBA) for the text encoder to enhance the relevant background perception ability in the diffusion generation process, without the need for specific background prompts. Meanwhile, we develope the Iterative Data Refinement with Self-improved LMM (IDR-LMM), a framework that iteratively enhances the data selection capability of LMM for diffusion model training, thereby yielding continuous performance improvements. 
Furthermore, we establish an E-commerce Product Background Dataset (EPBD) for the research in this paper and future work. 
Experimental results indicate that our approach significantly outperforms current prevalent methods in terms of automatic metrics and human evaluation, yielding improved background aesthetics and relevance.

Main paper: 
 introduction: 
 In today's digital landscape, captivating visual content is crucial for engaging audiences, especially in e-commerce. Aesthetically pleasing, high-quality product images have the power to captivate consumers and boost confidence in their purchasing decisions. However, product images uploaded by sellers may lack professional quality, exhibiting cluttered or plain white backgrounds. This paper explores the methods to enhance product image backgrounds, aiming to elevate their contextual appeal and effectiveness.Recently, text-to-image (T2I) diffusion models [16, 28, 31, 36 ] have demonstrated the ability to produce high quality visual content with textual prompts, which serve as the guidance of creativity of image generation. In order to guide the model in generating more desired content, crafting precise prompts with detailed instructions (e.g., objectives, style, limitations) is crucial. There has been a growing community of researchers and practitioners working on developing better prompt guidelines and techniques for generation effective prompts, in terms of generation stability, reliability and consistency [4, 14, 22, 23, 40] .However, designing prompts for e-commerce product image backgrounds is still challenging. It's hard to specify appropriate background elements in a prompt, such as lighting, scenery, styling, etc. Also, creating tailored prompts for a wide array of products to perfectly match the background with the product is impractical. In response to these issues, two categories of approaches can be considered: generating prompts with a fine-tuned language model [4] , and training T2I diffusion models to adhere to specific prompts [26, 45] .The first category introduces an extra language model, and such a pipeline may result in error accumulation and discontinuities in the gradient during the intermediate prompt generation process, making joint optimization challenging. This paper focuses on the second category, training end-to-end diffusion models solely using product names as conditions, without the complex prompt generation process. In specific, we propose Product2Img, a background generation diffusion model featuring Contrastive Background Alignment (CBA). This approach employs a contrastive learning objective to align the text encoder with the latent features of backgrounds. It eliminates the need for prompt design or engineering, making it a prompt-free method.By focusing less on designing prompts, the quality of images becomes the key factor in determining the model's outcomes. Several e-commerce product image datasets cover tasks such as image classification [11] , object detection [8] or image saliency prediction [19] , but they tend to offer limited product variety and overlook the aesthetic harmony between products and their backgrounds. Moreover, annotating image to meet e-commerce standards requires expertise and is labor-intensive. Recent research [1, 18, 49] highlights the effectiveness of large multimodal models (LMMs) in understanding images, suggesting a strong potential for image evaluation. However, LMMs' performance heavily depends on the specificity of their prompts, which makes it difficult for humans to include all necessary details for precise evaluations.Inspired by previous research [25, 39] that improve LLMs through feedback and prompt refinement, we design Iterative Data Refinement with Self-improved LMM (IDR-LMM) framework. It employs the LMM as an effective scorer to autonomously select images that meet e-commerce standards. To the best of our knowledge, this is the first implementation of self-improvement in an LMM for continuous data refinement during model training.To train and evaluate our method, we create E-commerce Product Background Dataset (EPBD), a high-quality aesthetic dataset with 25k product images with contextual background and corresponding product names. We conduct extensive experiments on EPBD, which demonstrates the effectiveness of our proposed method. We will release the dataset to facilitate further research.Contributions of this paper are summarized as follows:• We present Product2Img, a prompt-free method that employs Contrastive Background Alignment to generate appropriate backgrounds using only the product names, eliminating the need for complex prompting.• We design annotation-free IDR-LMM, an iterative data refinement strategy, to iteratively refine training data by improving the LMM through its own feedback on diffusion model outputs. This approach effectively enhances the quality of generation results. • We develop the high-quality E-commerce Product Background Dataset (EPBD). By releasing this dataset, we aim to encourage further research in this area. • We achieve superior product background generation for ecommerce with an effective end-to-end approach, ensuring both aesthetic and semantic compatibility. 
related works 2.1 text-to-image diffusion: 
 Text-to-Image Diffusion is a multi-modal task of generating images conditioned on texts. In the early years, popular image generation networks were mainly based on Generative Adversarial Network (GAN) [12] . Recently, diffusion models [16, 35] , such as DALLE-2 [30] , Imagen [33] , and Stable Diffusion [31] have achieved remarkable results. Text-to-image diffusion typically involves using text encoders, such as pretrained language models like CLIP [29] ,to encode text inputs into latent vectors. These vectors act as conditioning signals for the diffusion model, enabling the generation of images related to the text through a process of progressively removing the noise [31] .A common operation to generate backgrounds is combining the foreground (object) from one image with another background image to create a composite image [27] . Over the past years, researchers have focused on enhancing the realism of composite images by addressing aspects such as color harmony, lighting congruence, texture matching, and geometric alignment [6, 9, 38, 42] . As diffusion models are widely applied, Paint-by-Example [43] tackles this semantic image composition problem with an image-conditioned diffusion model trained in a self-supervised manner, where the reference image is semantically transformed and harmonized before blending into another image. However, these methods all require a background image as an input to serve as a reference.Image inpainting [2, 24, 37] emerges as a viable method for automated background generation, especially with the support of high-quality diffusion models like SD-Inpainting [31] , which significantly advance image inpainting technology by fine-tuning large-scale text-to-image pre-trained model. ControlNet Inpainting [48] offers image inpainting controlled through stable diffusion with additional control signals. PowerPaint [50] achieves better text-image context alignment by learning specific task prompts, leading to cutting-edge results in context-aware image inpainting and text-guided object inpainting. However, these techniques, while impressive, do not specifically focus on background generation and require detailed prompt engineering for precise descriptions."Self-improved" refers to the enhancement of LLMs using their own generated data. Research [3] has shown that large language models (LLMs) can be improved through careful prompt engineering and techniques such as In-context Learning [10] or Chain of Thought [41] . This is because a well-crafted context can effectively elicit the desired response from LLMs. More research [7, 13, 25, 44, 46] further explores the self-improvement capability of LLMs by gathering experience from past interactions and integrating it into future queries, thereby enhancing their performance. Recent work has shown that this capability allows LLMs to exhibit significant potential in improved learning mechanisms [17] and data annotation [47] . In this paper, we extend the concept of self-improvement to LMMs, enabling them to better select training data for diffusion model. 
dataset 3.1 data collection: 
 For the purpose of this study, we carefully curate a dataset of product images complemented by product names. To ensure image quality, 4 professional product image designers, diverse in age and gender, are hired to create high-quality images from scratch , adhering to a set of specific criteria. These criteria include the aesthetic appeal of the product image background, the relevance of the background to the product, and the avoidance of marketing text and logos within the product background.The dataset covers a wide range of categories, ensuring its comprehensiveness. To prepare the image input for our model, we employ an in-house image cutout tool to extract the product object from each image. Then we standardize the resolution of all images to 512x512 pixels. This professionally curated collection is named E-commerce Product Background Dataset (EPBD), serving as a crucial resource for our ongoing and future research projects.Collagen C Facial Cleanser Whitening Face Wash Foam Face Cream Skin Care Oil Control Collagen Moisturizing Black Spot EPBD comprises of 25,594 samples, each including a cutout of the product, product name in English, and an attractive product image featured a contextually relevant background. For analysis and model training purposes, this dataset is stratified into specific subsets: 24,394 samples for training, 200 for validation, and 1,000 samples  
method: 
 In this section, we first introduce the task definition and preliminaries on text-to-image diffusion models. Then, we explain the motivation and design behind our proposed Product2Img model in detail, which is demonstrated in Figure 4 and Figure 5 .The task centers on the generation of contextually coherent and visually appealing backgrounds for product images, known as Product Background Generation. The input for this task comprises a set of product cutouts 𝐶 and their corresponding product names 𝑃. Product cutouts refer to the images of products that have been segmented from their original backgrounds, the product names serve to provide additional semantic information, aiding in the comprehension of the product types and potential usage scenarios.The goal is to create output images 𝐼 that seamlessly integrate these cutouts into high-quality backgrounds, tailored to the product's intended environmental setting and consumer expectations, thereby enhancing the image's suitability for e-commerce.Diffusion models are a class of generative models that include two processes: the diffusion process (also known as the forward process) and the denoising process (reverse process) [16] . During the forward process, noise 𝜖 ∼ N (0, I) is added to the clean image 𝑥 0 as follows:EQUATIONwhere 𝑥 𝑡 is the noisy image at time step 𝑡, and ᾱ𝑡 indicates the corresponding noise level. Correspondingly, the denoising process involves a neural network U-Net [32] , parameterized by 𝜃 1 , denoted as 𝜖 𝜃 1 , which predicts the noise to generate an image from Gaussian noise. Furthermore, diffusion models can also be conditioned on other inputs [31] . For example, text-to-image diffusion models use text features encoded by a text encoder Γ 𝜃 2 to guide the diffusion process in the form of cross-attention, and are optimized using the following simplified objective function:EQUATIONwhere 𝑝 represents the text conditioned on the text, i.e., the prompt.CLIP [29] , a multimodal model trained on the large-scale WebIm-ageText dataset through contrastive learning, is able to understand the connections between text and images. However, the text in the dataset commonly overlook background details, making it challenging for CLIP to accurately align text with background elements. To further facilitate the comprehension of text-background matching, we devise a Contrastive Background Alignment (CBA) algorithm, which specifically learns the association between product names and the latent states of their corresponding backgrounds , thereby guiding the image diffusion model to generate appropriate background.To achieve this, we introduce a learnable background prompt < 𝑏𝑎𝑐𝑘𝑔𝑟𝑜𝑢𝑛𝑑 >, a special token denoted as 𝑏, which is initialized with the text feature vector corresponding to the term "background". It will be appended to the end of the product name 𝑃 to serve as the input for the CLIP text encoder. The hidden states outputted by the text encoder are then utilized as textual guidance conditions for the denoising U-Net. To fine-tune the diffusion model for background generation, we follow SD-Inpainting [31] by expanding the first convolutional layer of the denoising U-Net 𝜖 𝜃 1 to include five additional channels specifically designed for the product cutout 𝐶 and its corresponding mask 𝑚. The input is composed of a noisy image latent space representation at timestep 𝑡, the latent space representation of the product cutout, and the mask, concatenated together and denoted as 𝑥 ′ 𝑡 .L inpaint = E 𝐶,𝑚,𝐼,𝑃,𝑏,𝑡,𝜖 𝜖 -𝜖 𝜃 1 (𝑥 ′ 𝑡 , Γ 𝜃 2 (𝑃, 𝑏), 𝑡)EQUATION)To further enhance the product-background matching capability of the text encoder, we propose an auxiliary task based on contrastive learning to capture and leverage the latent correspondence between product names 𝑃 and product backgrounds 𝐵. Specifically, we treat the product name and its corresponding product background as positive samples, while pairing the product names with backgrounds of other products within the same training batch as negative samples. This approach involves minimizing the following loss function:EQUATIONr 𝑏 is the feature encoded by the text encoder and pooled through the background prompt 𝑏, and r 𝐵 denotes the image encoding features of the background image, extracted through a CLIP image encoder [29] . The term sim(r 1 , r 2 ) refers to the cosine similarity, which is defined as sim(r 1 ,r 2 ) = r ⊤ 1 r 2 ∥r 1 ∥ • ∥r 2 ∥. Furthermore, 𝜏 is the temperature hyperparameter, which is set by default to 0.05.Given the symmetry inherent in text-image contrastive learning, we define the symmetrized contrastive loss between the background prompt 𝑏 and background image 𝐵 asL ′ cl (𝑏, 𝐵) = L cl (𝑏,𝐵)+L cl (𝐵,𝑏 ) 2 .Additionally, following CLIP [29] , we also engage in contrastive learning between the product name 𝑃 and the image 𝐼 with symmetrized contrastive loss, denoted as L ′ cl (𝑃, 𝐼 ). Here, we use the output from the highest layer of the text encoder at the position of <eos> token as the feature of the entire sentence.The final loss function comprises the aforementioned multiple loss components, with the specific computation process as follows:EQUATIONwhere 𝜆 represents a hyperparameter to balance the contributions of the different loss terms. To simplify the training process, we assign the same weight on contrastive loss terms. It is noteworthy that performance might see further improvement through weight adjustment.This algorithm not only prevents catastrophic forgetting in the CLIP text encoder but also adapts the text encoder for the e-commerce domain.Self-improved LMM Using feedback 𝑓 𝑖 -1 to enhance the selection of the high-quality data subset 𝑆 𝑖 from D 𝑖 -1 by Eq.( 8 Use I 𝑑𝑒𝑣 to obtain feedback 𝑓 𝑖 to improve LMM by Eq. (7) 11:𝑖 ← 𝑖 + 1 12: end while 13: return M 𝑆 R 4.4.1 Data Selection. In this approach, a dataset D is first defined, which consists of triplets 𝑥 = {𝐶, 𝑃, 𝐼 }, representing product cutouts, product names, and product images with background. Then, a pretrained text-to-image inpainting model M = 𝜖 𝜃 1 , Γ 𝜃 2 is fine-tuned on D, noted as M D . Our target is to select a subset 𝑆 ⊂ D from D, so that the new model M 𝑆 trained on 𝑆 can achieve improved performance compared to M D .In specific, we use the industry-leading large multimodal model GPT4-Vision [1] as an automatic scorer in our IDR-LMM method. This scorer assigns a score 𝑅(𝑥, 𝑝 𝑅 , 𝑐 𝑅 ) to each sample 𝑥 ∈ D using preset prompt words 𝑝 𝑅 and selected samples 𝑐 𝑅 , with the accuracy of scoring improved by adopting techniques similar to Chain of Thought (CoT) [41] and In-context Learning (ICL) [3] . 1 Finally, samples 𝑥 𝑖 whose scores are higher than a certain threshold 𝛾 will be added to 𝑆, resulting in:EQUATIONwhere 𝑖 refers to the current iteration round, and update D 𝑖 ← 𝑆 𝑖 . Next, feedback 𝑓 𝑖 will be incorporated into the data scoring process to help select data more suitable for training. To be specific, we incorporate feedback on the generated images from diffusion model into the LMM query, enabling LMM to be more focused on the issues in model generation. The action process of an LMM can be considered as a Partially Observable Markov Decision Process [5] . In this framework, the training and evaluation of the diffusion model, viewed as the environment, yield feedback as observations. These observed feedback, articulated in natural language, informs the LMM's subsequent actions, thereby facilitating self-improvement.Thus the data selection equation 6 is updated to:EQUATIONThrough continuous iteration of these three steps until reaching the predetermined maximum number of iterations, this method can achieve continuous improvement in data quality and steady enhancement of model performance. [20] sampler is used. 
baselines.: 
 To validate the superior performance of the proposed method in the task of background generation, this section compares it with the latest, competitive, and available image inpainting methods as well as an object-driven generation approach. The methods selected for comparison and their brief introductions are as follows:• SD-Inpainting [31] : SD-Inpainting is a model that has been fine-tuned specifically for image inpainting tasks using Stable Diffusion, trained with random masks and image captions. • ControlNet-Inpainting (canny) [48] : Controls image inpainting using Stable Diffusion with a conditional encoder that encodes masks. To prevent the model from altering product shapes while generating product images, an additional canny ControlNet is incorporated to enhance the control and improve the results. • IP-Adapter [45] : An efficient and lightweight adapter designed to enable image prompting capabilities in pretrained text-to-image diffusion models. Unlike other baselines, this is an object-driven approach. Product images are used as image prompts and product names as text prompts to generate competitive images. • PowerPaint [50] : A high-quality, versatile inpainting model that has shown excellent performance. It achieves the best performance in multi-purpose image inpainting by tailored fine-tuning strategies, and multi-task learning. • EPBD-FT: EPBD-FT is a model initialized by SD-Inpainting and fine-tuned on the our EPBD dataset for product background generation.We establish a uniform prompt template across all baselines to ensure optimal and stable performance, detailed as follows: "{Product Name}, high-quality and appropriate background".Our objective is generating backgrounds from product cutouts and product names. The results should be realistic, aesthetically pleasing, and the background should be sufficiently relevant to the product. To evaluate from these perspectives, we employ the following metrics to assess the quality of the generated backgrounds.• FID (Fréchet Inception Distance) [15] : FID indicates the diversity and quality of generated images and is widely used to assess the outcomes of image generation tasks. A smaller FID indicates better image diversity and quality.• Aesthetics [34] : Aesthetics focuses on the aesthetic quality of images and can predict human ratings of image attractiveness on a scale from 1 to 10. A high score indicates that the image's aesthetics are recognized. • PickScore [21] : Grounded in rich user preference data, PickScore is built upon in-depth analysis and learning from a massive corpus of text-to-image samples, capturing the nuanced differences of human aesthetics. A higher PickScore indicates that the images generated based on specific prompts are more visually appealing and attractive to the audience. • CLIP-c-I [29] : Evaluates the relevance between the generated product image 𝐼 and the product subject image 𝑐. The higher the CLIP-c-I score, the greater the fidelity of the product in the generated product image. • CLIP-c-B [29] : Similar to CLIP-c-I, it evaluates the relevance between the product cutout 𝑐 and the generated background 𝐵. A higher CLIP score indicates a greater relevance between the product and its background in the generated image.To accurately evaluate the effectiveness of the proposed method, we conducted a systematic performance comparison between our approach and existing techniques on the test set of EPBD. Specific experimental results are detailed in Table 1 . As the table illustrates, our proposed Product2Img method demonstrates significant improvements across all evaluation metrics, which emphatically confirms the effectiveness of our approach.It is particularly noteworthy that even the strongest baseline "EPBD-FT", which adopted a similar fine-tuning strategy on the same training set as our method, also exhibited considerable performance improvement. This further validates the quality and practical value of the EPBD. However, in direct comparisons, our method outperformed on every performance metric. Especially in visual aesthetics (Aesthetics) and CLIP-based product-background consistency (CLIP-c-B), our method achieved improvements of 0.13 and 1.46%, respectively. These figures indicate that the CBA algorithm can successfully integrate product characteristics with background features, producing background images that more closely align with the product's style. Moreover, the iterative data refinement mechanism further optimizes the model at the data quality level, resulting in generated backgrounds that are not only more aesthetically appealing but also more harmoniously consistent in terms of product-background relevance. We also choose a variety of different products for qualitative evaluation of the proposed method, covering digital products, skincare products, backpacks, apparel, etc. The qualitative comparison in Figure 6 indicates that our proposed Product2Img achieves state-of-the-art performance in e-commerce product background generation in terms of aesthetics, product consistency, and the match between product and background. For example, regarding the children's backpack (1st row), olive essence (3rd row), and refrigerator (5th row), our method generates corresponding backgrounds of a child-style table, a plant backdrop, and a kitchen, respectively, demonstrating a high degree of relevance and aesthetic appeal to the products. On the contrary, the EPBD-FT method tends to position these products against less relevant or solid-color backgrounds. Meanwhile, other methods also struggle with issues like subject extension, and the generation of low-quality backgrounds and watermarks.From these cases, we also observe that our method is capable of generating a more diverse range of backgrounds which are not included in the EPBD training set, illustrating that our method can more effectively activate the background generation capabilities of the pretrained diffusion model.To conduct a more comprehensive comparison, we further engage in user studies. Specifically, we randomly extract 200 samples from test set and invite 5 experienced professionals to select their preferred results from those generated by different methods. We randomize the presentation order of the generated results to guarantee an unbiased evaluation. The professionals are given unlimited time to make their selections based on three criteria: visual appeal, consistency between the background and the product, and overall quality.The evaluation results are presented in Figure 7 . It is evident that our method achieves the highest winning rate across all three criteria, with a significant advantage in visual appeal. In summary, this user study suggests that our method is chosen as the most satisfactory solution, demonstrating the effectiveness of our approach.Consistency Overall Quality The effectiveness of CBA. To bolster the text encoder's ability to recognize the product background details, we design CBA which integrates a contrastive learning-based auxiliary task. We conduct an ablation experiment to assess its impact by removing this task and observing performance changes. The results in Table 2 show a deterioration in performance when CBA is not applied. For example, there is a decrease from 57.30% to 56.10% in the CLIP-c-B, which underscores the effectiveness of CBA in enhancing the model's ability to generate background that are relevant to the product. In product background generation models, data quality significantly impacts performance. We conduct experiments with IDR-LMM and thorough performance evaluations after each round. As shown in Table 3 , ablation studies show that model performance steadily improves across various metrics with each round, particularly in the CLIP-c-B metric, suggesting better consistency between the generated background and the product. However, from the second to the third round, there is a slowdown in performance gain, leading us to halt at round 3. Figure 8 illustrates the improvement in training data quality across various aspects during the Iterative Data Refinement process. LMM continuously improves its data filtration capabilities by incorporating feedback obtained from evaluating images generated by the diffusion model into its queries. It is inevitable that training on progressively higher-quality data will lead to continuous improvement in model performance. Moreover, control experiments confirm the necessity of IDR-LMM. By comparing the performance of models trained with and without IDR-LMM while maintaining consistent training steps, it becomes evident that data filtering not only elevates visual aesthetics but also enhances product-background matching. This indicates that the performance gains are primarily attributed to improvements in data quality, rather than mere increases in training duration. 
conclusion: 
 In this paper, we propose a prompt-free Product2Img diffusion model for end-to-end high-quality product background generation. To accomplish this, we design a Contrastive Background Alignment (CBA) algorithm to align the hidden features between product names and their image backgrounds, thereby enhancing the relevance of the generated backgrounds and products. Additionally, we propose Iterative Data Refinement with Self-improved LMM (IDR-LMM) to gradually improve training data quality, resulting in the generation of more aesthetic and realistic backgrounds. Furthermore, we construct the E-commerce Product Background Dataset (EPBD) as the foundation for our work and future research endeavors. Comprehensive experimental evidence demonstrates that our approach substantially surpasses current prevalent techniques in generating product backgrounds with superior aesthetics and relevance. Overall, Prodcut2Img provides an efficient and effective approach for product backgrounds generation, fulfilling a critical demand in the e-commerce domain. 
 
 Appendix: Additional implementation details and LMM prompts (𝑝 𝑅 , 𝑝 𝐹 , 𝑝 𝑆 ) can be found in the appendix, located within the supplementary material. 


Decision: The reviewers agree that this is an interesting and relevant paper that is well written. The experimental setup, including ablation studies and a user study, is well designed and demonstrates the effectiveness of the proposed method. However, the reviewers mention that the novelty is rather low and the results might be heavily impacted by the dataset used for training and that the method may suffer from potential overfitting due to the rapid reduction in training samples. In this context the reviewers question the method's effectiveness for open-domain items and point out that the method does not seem to work for transparent products. However, the authors responded appropriately in their rebuttal and could clarify some of the mentioned drawbacks. In particular they show that their method does not overfit too strongly and that it is still useful for transparent product images. Considering all reviews and ratings, as well as the rebuttal of the author, I conclude that this work should be accepted.

Reviews: review: paper_summary: paper_summary: The paper presents a method to generate background images suitable for the presentation of products for e-commerce.
The idea is that often generative models are not able to capture the semantics of the background, that interplays with the semantic of the product itself.
The method doesn't require specific prompt for the background, and follows an iterative approach for training.
In addition a novel dataset is proposed.

The core elements of the proposed approach are:

-  a novel dataset (EPBD) of products/descriptions and backgrounds, curated by 4 professionals (that I expect/assume will be made available if the pair gets accepted)
- a diffusion model extended to have additional input consisting of product mask, product cut
- a training process that combines an iterative "self-improving" process (IDR-LMM) and a loss that accounts for better alignment of product image and background (CBA)
strength_weakness: strength_weakness: The task is interesting and quite relevant for the multimedia community.

The paper is well written, it's easy to understand the method and only minor presentation improvements may be required.

The experimental setup seems to be well designed, with ablation studies and a user study that helps to make more sense of the quantitative scores in Tab. 1. The user study shows that the images generated by the proposed approach are preferred over the competing baseline.
limitations: limitations: The experiments, as noted also by authors in Sect. 5.2, show that the dataset proposed is fundamental to obtain a good performance, since the EPBD-FT is basically a SD-Inpainting fine tuned on the dataset, with some very minor improvements when considering FD and PickScore metrics. This shows that the Contrastive Background Alignment and the iterative refinement parts of the proposed system may have a relatively reduced impact, perhaps simply increasing the size of the dataset could be enough (although the CLIP-c-I and CLIP-c-B metrics register slightly more significant improvements).
Similar tiny improvements are shown also in the ablation regarding the iterative data refinement part of the approach as shown in tab. 3, again for metrics other than CLIP-c-B, hinting that the effectiveness fo the system is more dependent on the dataset and probably on the loss that aligns CLIP with the background (CBA).

Overall the experiments show that the really important part of the system is CBA, thus reducing the novelty of the proposed system.

Improving the presentation: introduce the LMM acronym the first time it's used on page 1, not on page 2.
The terms used to describe the design of the system in Sect. 4.2 and 4.3 (e.g. \theta_1, \epsilon_theta_1, mask m, cutout C, should be used in Fig. 4 and 5, to ease parsing text and figures. Correct the typo in the name fo the system on line 925!
The bibliography should introduce some papers form the MM community. For example, in ACM MM several papers regarding generative approaches for fashion have been presented, some may be cited in this work.
paper_summary: paper_summary: The paper proposes a prompt-free diffusion model with automatic training data refinement strategy for product background generation. The authors also develop an E-commerce Product Background Dataset (EPBD). Experimental results indicate that the proposed method is better than compared methods.
strength_weakness: strength_weakness: The paper is written well and easy to follow.
The idea of this paper is interesting, that is generating background for products.
limitations: limitations: The proposed method seems to not work for transparent products.
How to know what background is suitable for a product? From Fig.6, it's hard to know which method is better than which method.
paper_summary: paper_summary: Using generative models to design visually appealing background images for products holds immense potential for quickly capturing consumers' attention. However, the reliance on text-guided image generation poses challenges, as it necessitates careful crafting of input text, hindering background automation and adaptive generation. To address this, the paper introduces the Product2Img model, featuring an automatic training data refinement strategy tailored for product background generation sans specific background prompts. Simultaneously, the paper presents the IDR-LMM framework, aimed at iteratively enhancing the data selection ability of Large Language Models (LLMs) in training diffusion models. Additionally, the authors curate the E-commerce Product Background Dataset (EPBD) to facilitate this study and future research endeavors. Experimental results demonstrate that Product2Img surpasses current popular methods in both automatic measurement and manual evaluation. Notably, it achieves enhanced background aesthetics and relevance, showcasing its significant superiority.
strength_weakness: strength_weakness: 1. The Product2Img model utilizes Contrastive Background Alignment to generate suitable backgrounds solely based on the product names. This innovative approach eliminates the need for textual descriptions of background design, thereby substantially reducing the overhead associated with prompt design for automated background generation.
2. The paper introduces the annotation-free IDR-LMM, an iterative data refinement strategy aimed at enhancing training data quality by iteratively improving the Large Language Model (LMM) through its feedback on diffusion model outputs. This method enables the evaluation of a high-quality subset of the dataset and facilitates continuous enhancement of model training quality. Such an approach holds significant reference value for subsequent model training endeavors.
3. The EPBD dataset introduced in this paper boasts rich classification and high image quality, representing a notable advancement in the domain of commodity background filling tasks. Its comprehensive nature and superior image quality hold significant reference value for future research endeavors in this field.
limitations: limitations: 1. The IDR-LMM strategy outlined in this paper holds considerable promise for data screening and subsequent training procedures. Figure 8 visually illustrates that as training progresses, there's a noticeable reduction in the quantity of data within the subset contributing significantly to training, accompanied by a marked enhancement in the CLIP index, albeit with only marginal improvements in FID. This phenomenon prompts inquiry into whether such rapid reduction in training samples could inadvertently lead to unforeseen overfitting. Moreover, questions arise regarding the method's efficacy in adequately complementing backgrounds for Open-Domain items beyond the dataset's confines. These aspects warrant further investigation and clarification to comprehensively understand the implications of the strategy.
2. The background alignment strategy based on contrast learning, as adopted in this paper, appears to mirror the setup utilized in DreamBooth, employing special tokens to learn specific semantics, in this case, supplementary image background information.  While DreamBooth excels at learning concrete concepts, it struggles with abstract concepts such as background, a limitation that this paper attempts to mitigate to a certain extent through fine-tuning the learning constraint diffusion model.
3. Additionally, this paper introduces aesthetic indicators such as PickScore and Aesthetic.  However, the proposed method does not directly optimize image aesthetics.  As reflected in Table 2 and Table 8, there is minimal improvement in aesthetic indicators.  The perceived aesthetic superiority of Product2Img appears to stem from the high-quality EPBD dataset.  This aspect warrants further discussion and exploration.