Title: Salient Concept-Aware Generative Data Augmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generative Data Augmentation

Fine-grained Image Classification
Abstract: Recent generative data augmentation methods conditioned on both image and text prompts struggle to balance between fidelity and diversity, as it is challenging to preserve essential image details while aligning with varied text prompts. 
This challenge arises because representations in the synthesis process often become entangled with non-essential input image attributes such as environmental contexts, creating conflicts with text prompts intended to modify these elements.
To address this, we propose a personalized image generation framework that uses a salient concept-aware image embedding model to reduce the influence of irrelevant visual details during the synthesis process, thereby maintaining intuitive alignment between image and text inputs.
By generating images that better preserve class-discriminative features with additional controlled variations, our framework effectively enhances the diversity of training datasets and thereby improves the robustness of downstream models.
Our approach demonstrates superior performance across eight fine-grained vision datasets, outperforming state-of-the-art augmentation methods with averaged classification accuracy improvements by 0.73\% and 6.5\% under conventional and long-tail settings, respectively.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of balancing fidelity and diversity in generative data augmentation (GDA) methods conditioned on both image and text prompts. The key idea is to use a salient concept-aware (SCA) image embedding model that emphasizes class-relevant features while suppressing irrelevant background or context information. The method first independently trains domain-specific salient concept-aware embedding model with a penalty to improve both the intra-class compactness and inter-class discrepancy. Then the method trains the synthesis model, taking frozen image and text embeddings as inputs and only tunes the adapters. This approach does not require intensive subject-specific optimization or additional input processing such as saliency segmentation. Experiments on eight fine-grained visual categorization datasets show improved classification accuracy compared with prior GDA methods, particularly in challenging long-tail and out-of-distribution scenarios. Extensive ablation studies further demonstrate the advantages and stability of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper is clearly written and provides valuable high-level insights. The figures in the main text significantly aid understanding, and the supplementary material offers comprehensive implementation details.\n\n- The work identifies a practical and impactful limitation of existing GDA methods—namely, that their embeddings often entangle information that is less relevant to the concept of interest.\n\n- The proposed two-stage training strategy, which first finetunes a domain-specific salient concept-aware embedding model and then trains the synthesis model, is effective. The use of a novel penalty to improve both intra-class compactness and inter-class separation during embedding model finetuning is particularly interesting. The design of the conditional synthesis model and image prompt adapter is also well-motivated, and the paper provides clear strategies for constructing and filtering training data.\n\n- The experiments are thorough, with extensive comparisons across multiple datasets demonstrating state-of-the-art performance. In addition, detailed ablation studies convincingly validate the effectiveness of each component.\n\n**Weaknesses**\n\n- The proposed approach still requires finetuning both the salient concept-aware embedding model and the synthesis model for each new dataset, which limits its generalizability and falls short of achieving truly optimization-free general data augmentation.\n\nQUESTIONS:\n- In the introduction, the paper refers to synthetic image generation as a form of distillation. Based on this, can generative data augmentation be understood as a process where knowledge from the salient concept-aware embedding model is distilled into the classifier via the synthesis model? From this perspective, I am curious whether using the features from the salient concept-aware embedding model directly for classification would yield results comparable to those obtained through generative data augmentation.\n\n- In the comparisons for conventional, long-tail, and OOD classification, is the amount of synthetic data generated by each method kept consistent? If not, I would appreciate it if the authors could report the amount of synthetic data produced by each method.\n\nLIMITATIONS:\nNo. The authors should include technical limitations and potential social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an improved method for generative data augmentation that focuses on generating diverse images of the same class while preserving only the relevant class features, while discarding unimportant information, such as the background or position.\nThe method is based on fine-tuning a diffusion model that is conditioned on \"salient concept-aware\" image embeddings, allowing the model to generate diverse images of the same input class.\nThe method is validated through different evaluations, including conventional and few-shot classification.\n\nSTRENGTHS AND WEAKNESSES:\nThe method is simple and effective in classification tasks. \nHowever, the method is expensive to train; IIUIC, it requires fine-tuning of the SCA and the diffusion model for each classification task.\nComparison to other off-the-shelf adapters is required.\nAdditionally, I found the scope of the method to be limited: could the proposed data augmentation help to improve generative personalization tasks (e.g., 1, 2), or other discriminative tasks?\n\n[1] An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion, Gal et al.\n[2] Multi-Concept Customization of Text-to-Image Diffusion, Kumari et al.\n\nQUESTIONS:\nSome things are not completely clear:\n- Is the Domain Expert Embedding Model trained together with the diffusion model? \n- Is it trained separately for each task?\n- L. 128-131: Why CLIP Image features are not suitable for the task? Unlike DINO, CLIP is trained under contrastive settings between images and **text** such that it captures the important concepts in the image.\n- I wonder how the method improves over augmentation that utilizes other pre-trained diffusion adapters, e.g.:\n[1] IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models, Ye et al.\n[2] BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing, Li et al.\n[3] ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation, Wei et al.\n\nLIMITATIONS:\nMentioned limitations were not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an image generation framework for data augmentation on image and text prompts. The main component of the design is a \"salient concept-aware\" image embedding model. The idea is to preserve key attributes of the input image while reducing the influence of the background features. This is achieved by training the image embeddings with additional penalty terms. Experiments are conducted on eight fine-grained image datasets, and the proposed method is compared with other data augmentation methods.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Generative data augmentation is an important task. Trying to address it to generate fine-grained categories is a good motivation. \n- The experiment section is fairly comprehensive, covering a wide range of methods and benchmarks, as well as essential ablations. \n- The paper focuses on obtaining better image embeddings, without the need to make substantial changes to the generation model itself. This makes the approach flexible, and it is possible to adapt to other generation models. \n\n### Weaknesses\n- The core of this paper is the loss term in the Salient Concept-Aware Embedding Model (Eq. 1). It looks like a tuned version of a contrastive loss (based on cosine similarity with margins) in a contrastive learning framework. It is unclear to me where the 'salient' part comes from in this formulation: I think it is closer to a max-margin positive feature vs negative feature learning scheme. \n- The 0.73% accuracy improvement in the conventional setting is underwhelming. One would expect that having a good data augmentation to obtain more diverse data should improve the results more, even in the traditional setting without the focus on the few-shot or long-tail cases.\n\nQUESTIONS:\n- Figure 2: for image (c), it seems the proposed model SCA learns some sort of style transfer--for the given image (f), it treats the black mask as the texture of the fly and draws a black fly/moth in image (c). This is slightly different from what is expected if the learned embeddings are truly about either the fly or the background flowers. Can the authors clarify what's happening here when a salient object is masked out? \n\n- Please see the discussion on Weaknesses and clarify how the proposed loss term enables the learning of 'salient' features. \n\n- In some experiments (such as Section 4.3), only a small subset of the dataset is selected. Is there a reason not to test on all fine-grained datasets? \n\nOverall, I like the simplicity of the approach and, in contrast, how much improvement it can achieve in the long-tail classification tasks. It shows promising results that might inspire more future research in addressing data augmentation for long-tail classes.\n\nLIMITATIONS:\nNo discussion on limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper presents a framework for generative data augmentation (GDA) that effectively addresses the fidelity-diversity trade-off by introducing a salient concept-aware (SCA) image embedding model. The proposed method disentangles essential visual features from irrelevant attributes, enabling better alignment with text prompts while preserving class-discriminative details. The authors demonstrate significant improvements in classification accuracy across multiple fine-grained datasets, particularly in long-tail settings.

STRENGTHS AND WEAKNESSES:
Strengths:

1. Proposes a novel salient concept-aware (SCA) embedding model that effectively disentangles essential class-discriminative features from irrelevant image attributes (e.g., background), directly addressing the fidelity-diversity trade-off in generative data augmentation.
2. Eliminates reliance on manual interventions (e.g., segmentation masks) or subject-specific optimization, enhancing scalability for real-world applications.
3. Demonstrates state-of-the-art results across eight fine-grained datasets under diverse settings (conventional, long-tail, few-shot), with significant accuracy gains (up to +6.5%).

Weaknesses:
This paper does not seem to have any major flaws.
1. Focuses exclusively on fine-grained visual tasks; applicability to broader or non-visual domains remains unverified.
2. Fails to analyze computational costs of training the SCA embedding model, a critical factor for adoption.

QUESTIONS:
According to the W.1-2.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-04