Salient Concept-Aware Generative Data Augmentation
Authors: Tianchen Zhao
Xuanbai Chen
Zhihua Li
Jun Fang
DONGSHENG An
Xiang Xu
Zhuowen Tu
Yifan Xing
Venue: NeurIPS 2025 poster ()
Keywords: Generative Data Augmentation
Fine-grained Image Classification
Primary area: deep_learning
TL;DR: We propose an effective data generation method using diffusion model to improve performance of downstream classifiers

Abstract: Recent generative data augmentation methods conditioned on both image and text prompts struggle to balance between fidelity and diversity, as it is challenging to preserve essential image details while aligning with varied text prompts. 
This challenge arises because representations in the synthesis process often become entangled with non-essential input image attributes such as environmental contexts, creating conflicts with text prompts intended to modify these elements.
To address this, we propose a personalized image generation framework that uses a salient concept-aware image embedding model to reduce the influence of irrelevant visual details during the synthesis process, thereby maintaining intuitive alignment between image and text inputs.
By generating images that better preserve class-discriminative features with additional controlled variations, our framework effectively enhances the diversity of training datasets and thereby improves the robustness of downstream models.
Our approach demonstrates superior performance across eight fine-grained vision datasets, outperforming state-of-the-art augmentation methods with averaged classification accuracy improvements by 0.73\% and 6.5\% under conventional and long-tail settings, respectively.

Salient Concept-Aware Generative Data Augmentation
Tianchen Zhao, Xuanbai Chen, Zhihua Li, Jun Fang, Dongsheng An, Xiang Xu, Zhuowen Tu, Yifan Xing
AWS DS3
Abstract
Recent generative data augmentation methods conditioned both image and text prompts struggle to balance between fidelity and diversity, as it is challenging preserve essential image details while aligning with varied text prompts. This challenge arises because representations in the synthesis process often become entangled with non-essential input image attributes such as environmental con- texts, creating conflicts with text prompts intended to modify these elements. To address this, we propose personalized image generation framework that uses a salient concept-aware image embedding model to reduce the influence of irrelevant visual details during the synthesis process, thereby maintaining intuitive align- ment between image and text inputs. By generating images that better preserve dineiiiscrei features with additional controlled variations, our framework effectively enhances the diversity of training datasets and thereby improves the robustness of downstream models. Our approach demonstrates superior perfor- mance across eight fine-grained vision datasets, outperforming state-of-the-ar augmentation methods with averaged classification accuracy improvements by 0.73% and 6.5% under conventional and long-tail settings, respectively.
Introduction
Text-to-image (T2I) models 63 68 have demonstrated remarkable success in generating high-fidelity
images. One promising application using synthetic image generation as a form of distillation,
transferring knowledge into classifiers to improve their performance [69| 5 75 97 [4]. However, T2I models often struggle to generate fine-grained categories due to the limitations of textual descriptions, introducing noise to the training set that can negatively impact the downstream classifier training. For example, ambiguous terms can lead to incorrect image synthesis, such as jaguar referring to either animal or Expert terminology outside the model's training vocabulary, such class names from iNaturalist dataset, may produce uncxpected output. Morcover, certain visual attributes, such human identities, cannot be fully captured through text alone. Generative data augmentation (GDA) (99] addresses these limitations by conditioning synthesis on both text and image to better capture the subject characteristics.
Specifically, GDA leverages personalized image generation methods (66] to augment existing images of given subject based on textual descriptions. However, existing approaches 776
rely on subject-specific optimization, which overly emphasizes instance-level details from the image prompts, limiting diversity when text prompts call for meaningful modifications. They also inherit issues from foundational tools such as DreamBooth (65) and Textual inversion [25], which are prone to overfitting suffer from imprecise subject representations. On the other hand, approaches such SuTI [14] provide more flexible alternatives, which are pre-trained on web-scale data and support zero-shot synthesis. However, these models often struggle to preserve the key concept in image that defines its class, especially when the class definition is abstract or the object corresponding to the target concept the image is small. This is caused by the inability of the methods to entail prior knowledge on what to focus on in the image. We are motivated to learn a specialized synthesis model tailored for
37 g3
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Spared let, denest
Japanese painting o bollyylnn, prvcr
ledt
wetower para
<lacy un melly
Lodybbug In s000s Sa dettiad
Permoi of bodybuu hing or teee praysosls dial
rafndrag cowared ptom
reds, mbt
Image Prompt
DA-Fusion
Figure 1: Comparison between our proposed data augmentation method with DA-Fusion [76.. DA-Fusion uses Textual Inversion [25] learn the word embedding of the ladybug from the image prompt. But in this example, the generated outputs skew heavily toward green color palette, failing to both preserve the ladybug's characteristics (black body with red spots) and align with the text prompts. Our method is designed with in-context understanding of the task: it selectively encodes salient concept (the ladyhug) and effectively reduce the influence of irrelevant information (the greens and leaves) from the prompt, generating images with focused diversity and thereby effectively improves the robustness of the downstream models.
target domain that understands comain--ppciffe characteristics, intra-class variations, and inter-class relationships. This allows the model to generate novel, faithful, and diverse images that enrich the distribution of the original dataset, particularly for underrepresented or unseen categories, improving the robustness of the downstream classifier.
To this end, we define the problem of domain-specific personalized image generation for GDA: con- sider a joint distribution (X, W), where each sample consists of an image x; E X and a corresponding fine-grained label y: Y, our goal is generate image x' E X that accurately captures the key concept about Y/ from the image prompt 24, while also aligning with some relevant text prompt t; that describes yt. We adopt the framework [4 [9 that uses an image embedding model to capture the concept of interest from image. which is then integrated with text embedding to control a diffusion model. While originally designed for personalized image generation
86 26
38 29 58).
this approach has not been explored for GDA applications, particularly in properly balancing the addlity-diversipg trade-off.
Our insight is that the widely adopted practice of naively using image embeddings to control generative model is inadequate for GDA: these embeddings often entangle information that is less relevant to the concept of interest, restricting the model's ability to produce faithful results aligning with text prompts. To mitigate, we propose to train a salient concept-aware (SCA) image embedding model with the loss function that learns discriminative representation space with high intra-class compactness, mitigating the entanglement of less relevant information, and inter-class separation, extracting distinctive salient concept characteristics from image prompts. As an example shown in Figure I embeddings learned from Textual Inversion capture irrelevant background information from the image, which conflict with the text prompt that asks for different background. As result, the generated images cannot accurately reflect the variants asked by the text prompt. In contrast, our generative model with salient concept-aware embedding focuses on the main subject and can generate images faithful to the subject. In addition, our approach more convenient than methods that rely on additional input processing to capture target concepts, such saliency detection or segmentation [42]. These approaches typically require additional human interventions to make sure the segmented concept is relevant to the task, and are limited to scenarios where the target object can be clearly defined with spatial boundaries 15| 88
Our contribution is that we are the first to leverage the recent personalized image generation framework with dual embedding models for generative data augmentation, without the need for subject-specte optimization. The proposed framework mitigate the adelity-diversin trade-off by training salient concept-aware image embedding model that captures the salient concept of the image while adapting effectively to diverse text edit instructions. Our approach is evaluated on concept preservation, text alignment, and downstream classification accuracy across conventional, few-shot, long-tail, and out-of distribbitin settings. Experiments on eight Fine-Grained Visual Categorization (FGVC)

---

datasets [7 56 51 79 43 41
demonstrate state-of- of-the-art performance over existing generative data augmentation methods. Specifically, our method improves classification accuracy by 0.73% in conventional settings and 6.5% in long-tail settings.
2 Related Work
Image generation has made remarkable progress with the success of foundation T2I diffusion models
[55 62 63 600 20, 68, 6, 87 19 24,
90]. Moreover, their adaptability to incorporate additional
controls has drawn significant interest 995 98
52
54 48
40
30, 36,
67 71 80], which are trained
on paired image-text data at web scale but they struggle to establish concepts that are less frequently appeared in the training set or cannot be described by language clearly.
Personalized Image Generation. Different from image-editing approaches, personalized image generation faithfully reflects specific concept in novel scenes, drawing inspiration from reference images from personal lives. Given a few samples of a concept, DreamBooth [65] fine-tunes the full model and Textual inversion [25] optimizes word vector for the new concept, followed by
subsequent works generalized to muiltiple concepts 44 27
86 To support broad adoption for
practical usage, training-free methods 14 89
70
26
38
15
13
3
support zero-shot
synthesis using reference images. These methods train single model with surge-scale database instead of per-object optimization, and do not necessarily preserve exact instance details from image prompt. However, users often observe suboptimal performance when working with these models, which are not specifically tailored to their categories of interest.
Synthesis for Analysis. Traditional methods such as Mixup 194] and CutMix 93 transform ex- isting data but cannot introduce genuinely novel visual content. Recent efforts have focused on fine-tuning T2I models on in in-domain datasets to improve fidelity (3) or leveraging prompt en- hancement techniques [92] 7 45 improve diversity. However, the scalability of synthetic data for classification remains unsatisfactory 28 91, 21). as generated samples often exhibit limited transparency in their creation process, struggle to generalize across different contexts, and frequently require prompt engineering to obtain faithful results. Generative data augmentation (GDA) tech- leverage personalized image generation improve the fidelity of 76 utilizes SDEdit [52] and textual inversion to modify real samples with controlled generation strength. DiffuseMix (37 combine cut-mix with pix2pix [0] generated images. Diff-Mix (83) combines T2I personalization with 12I translation. Existing methods mostly use combination of popular tools to edit upon real images by initializing diffusion with the images instead of noises, constraining image editing to limited set of disentangled attributes such as textures, backgrounds, and colors. Our approach implements an end-to-end framework that encodes relevant image information into embedding, supporting concept- peneept-rresevvin image generation through free-form text editing. As a result, our method produces faithful, diverse training data that better challenges and improves the robustness of classifiers.
niques [99 76 102 53
synthetic data, DA-Fusion
3 Our Approach
Generative data augmentation is challenging task that aims to create diverse, high-fidelity datasets that improve the robustness and generalization capabilities of downstream classification models. Given a text-to-image generative model, a joint distribution (X, Y) representing the classification task, our goal is to fine-tune a domain-specific generative model G so that, given image z; and a relevant text prompt t,, the generated image (z,, t,) satisfies: 1) concept consistency, where the generated image G(z,, t.) retains the key semantic attributes of 2,, and 2) text consistency, where ((x,, tx) accurately reflects the meaning t,.
Our main contribution lies in demonstrating the significance of the image embedding used for training synthesis model in achieving an optimal trade-off between text and image prompt consistency. Specifically, we divide our training into two stages. First, to achieve stronger supervision and better data efficiency, we independently train domain-specific salient concept-aware embedding model (SCA), as introduced in Section 3.1 Then, we train the synthesis model, taking frozen image and text embeddings as inputs and only tune the adapters, as described in Section (3.2 We then explain our data synthesis approach for generating downstream classification training data in Section 3.3
3

---

Ours: SCA
Ours: SCA Embedding
Image Prompt
DINOV2 Embedding
DINOv2 Synthetic Image
Synthetic Image
SGA Embedder
SEg
Backbone
G
SEg
Da
DINOv2
Backbone
sim: 0.82
cos sim: 0.25
(b)
(h)
Generation Model
SEg
Dg
sim: 0.06
cos sim: 0.83
Key Feature
Mollag
(fly)
SEg
Irrelevant Feature (flower, leaf)
SCA focuses on salient concept of interest
DINO preserves inrelevant background info
Figure 2: Comparison of generated images conditioned on various image prompts and null text prompt. Our model focuses the fly in the image, evidenced by the high cosine similarity 0.82 between the segmented fly image (d) and the original fly image (e), allowing Foundation model like DINOv2 encodes lots information similarity 0.83 between the embeddings of the original fly image (c) and the image with the fly removed (f), from which the model can reconstruct the background (i). Although DINOv2 works well on segmented images with the salient object to generate image (g), it introduces additional complexity the pipeline, which is not suited for data augmentation purposes, and does not work well with abstract concepts without well-defined boundaries.
synthesis model generate high-fidelity fly images (ab). the background, evidenced by the high cosine
3.1 Salient Concept-Aware Embedding Model
Previous personalized image generation methods guide the synthesis with text and image feature extracted from large-scale pre-trained models such as CLIP [61] or DINO [1] [7]. However, as shown in Figure 2 those image features entangled with irrelevant information for the salient concept from the prompt image, confliciig with the information provided by the text prompt [89].
Naive adoption of embeddings from a pre-trained classifier results in significant performance degradation, as cross-ent cross-entropy (CE) does not explicitly encourage discriminative capability under large intra-class small inter-class appearance variations [18), and does not generalize well for underrepresented categories. As we demonstrate our ablation study, data generated under the guidance of a classifier's 's embeddings provides marginal improvement for downstream training.
In this work, we introduce penalty to improve both the intra-class compactness and inter-class discrepancy. Specifically, given samples 2, and æę from class p, and v, from a different class g. where p. q E Y, we enforce the condition DJF f2) < D(J f9), where is some distance measure in the embedding space and f. fh. and f are the respective embeddings, i.e., f = F(x). To this end, adopt the margin function:
RENN Σ
Term; = e soco (areooaco f.)+t), Termy
eo-W?" Is
q 1,97/1 Li
1 N
Termi
Lmangin
Σ log
(1)
Term + Term
where W, E Rdd denotes the p-th column of the weight matrix W E d is the embedding vector size, [Y is the number of classes, N is the total data volume, WT fi represents the logit for the sample Ie, and variables e and s denote the margin penalty and scale factor, respectively. In practice, we normalize each individual weight W, and embedding feature f. in fz. By adding an additive angular margin e to the target angle, the embedding model J avoids encoding confounding characteristics from different classes and focusing more extracting key concept of interest. As shown in Figure 2 the salient concept-aware embedding focuses on the main subject and can condition the generative model to output images faithful to the subject from image prompt.

---

Enc
U-Net - LoRA
Dec
Domain Expert
Embedding Model
Aftn
x4
Proj
Sleppy feline snuggled, knitted blaniet
Text Encoder
Figure 3: Our proposed framework. The SCA embedding model captures the salient concept and projects it with text embedding into cross-attention layers U-Net. LoRA applicd self-attention layers.
Synthesis Model
Architecture. Our salient concept-aware (SCA) embedding model trained in the first stage extracts visual features from image prompt, which are then projected into feature space with the same dimensionality the text features in the pretrained diffusion model. Similar to the text features C projected image features Cring are fed into every cross-attention layer in the U-Net, with new key and value projectors. The attention Z is calculated text and the image embeddings, with a weight factor set to the default value I:
weighted sum of the attention from both the
Z = "QK"") + X(QKime)
(2)
Here, Kimg, and Ving are the key and value projections for the image prompt, defined as: Wime
K iing
Vimg = Cimm
where winm and wii are weight matrices for key and value projections. respectively. For better adaptation with image embeddings, we apply LoRA [35] to the text branch:
Q D= + AWQ k = W + AWW Vou = WIy AWpIO (4)
where 7prev is the previous state of the diffusion model, C denotes text prompt feature, Wo. W and WP7 are the weight matrices and AWq. AWR. K and AWR are the corresponding LoRA linear layers, for the query, key, and value projections, respectively. We also apply LoRA to all self-attention layers of the U-Net in similar fashion for better domain adaptation. To bridge between the embedding spaces of images and text, we adopt weeght image embedding projector [47] consists of four building blocks, each comprising cross-attention layer and fully connected layer with residual connections. See Figure 3 for more details on the synthesis model architecture.
Multi-Modal CFG. We apply classifier fassi free guidiance by randomly dropping image text prompt control during training. In particular for image prompt, instead of using zero vectors, we utilize the average embedding of training data as the class-conditioning input for unconditional models.
3.3 Synthesis for Analysis
Data Construction. For the training data used to fine-tune the adapter and LoRA, generate corresponding prompts using image captioning models such as Blip2 [47] and MiniGPT [03] For the inference data used to train the downstream classifier, separate large language model to produce contextualized descriptions of images featuring a specific class. These descriptions combine foreground objects and background elements in concise manner. An illustrative example is provided below. Note that for both training and inference text prompts, randomly replace the fine-grained class names with meta class names for robust training and diversified inference, e.g., replace "bulldog" with "dog". To generate a synthetic data of given category, randomly sample real data from that category along with the corresponding LLM prompt to condition our synthesis model.
prompt Slaud-3-Soomer 2
Question: Imagine there is photo of [CLASS]. Describe the photo with short sentence, followed by a few relevant descriptive terms about the background of the photo, separated by commas. This is used to prompt text-to-imant models.
Answer: [Baby] A cute baby lying on a soft blanket, nursery, pastel colors, toys.
[Crane] A majestic crane stands tall, serene lake, lush greenery, distant mountains.
5

---

Backbone
Method
CUB
Aircralts
Flowers
Cars
Dogs
Food
Avg.
Vanilla
86.62
89.14
99.29
94.47
87.15
92.01
91.45
CutMix 93
87.31
89.19
99.21
94.74
87.57
92.59
91.77
Mixup
94
86.74
89.43
99.49
94.41
87.49
91.45
91.50
RN50
Real-Guidance 28
87.26
89.21
99.26
94.64
87.27
92.03
91.61
Da-I'usion 76
86.47
87.96
99.31
94.69
87.36
91.59
91.23
Diff-Mix
87.50
90.12
99.44
95.21
87.88
92.21
92.06
Ours
89.68
91.31
99.59
95.12
88.01
94.12
92.97
Vanilla
89.94
85,46
99.53
94.23
94,11 11
92.09
CutMix [6
91.09
85.44
99.61
94.85
91.94
95.62
93.08
Mixup
[X]
90.89
86.27
99.70
94.87
91.83
94.17
93.12
ViT-B/16
Real-Guidance
90.11
85.13
99.56
94.67
91.86
93.59
92.49
Da-Fusion
89.97
83.84
99.58
94.55
94.23
92.01
Diff-Mix 83
91.13
88.07
99.74
94.93
91.76
93.53
93.19
Ours
91.47
88.42
99.92
95.17
91.84
95.61
93.74
Table I: Conventional classification Top-1 accuracy (%) across six datasets using RN50 and ViT-B/16. We achieve an average improvement 0.73% over peer methods.
Model
Method
IN
Aircrafts
Cars
Food
Flowers
Avg.
Vanilla
69.61
58.25
87.74
87.62
98.80
80,40
CutMix 93
70.93
61.27
89,05
88.93
98.97
81.83
Mixup X4
71.24
62.39
89.07
87.59
99.07
81.87
VIT-B/16
Du-Fusion 76
71.95
65.71
90.94
88.01
99.25
83.17
Diff-Mix
83
70.84
63.51
89.74
88.46
99.09
82.33
Ours
72.41
72.44
92.72
88.38
99.17
85.02
Table 2: Few-shot classification Top-1 accuracy (%) using 10-shot training samples across five datasets.
Filtering. We use CLIP assess text coherence by computing the cosine similarity between embeddings of text prompt and generated image. To evaluate salient concept coherence, we measure the cosine similarity between SCA embeddings of image prompt and generated image. Data samples with the lowest 10% in either text or salient concept consistency are filtered out.
Classifier Training. We pre-generate and filter synthetic training data, producing final dataset that is of roughly K times the size of the original training set. During classifier training, each batch combines synthetic and real samples, where P% of the data are sampled from synthetic pool.
Experiments
We introduce the experimental settings in Section We compare our method with GDA baselines in Section 4.2 Ablation studies and qualitative analysis shown in Section 4.3 More results are provided in the Supplementary.
4.1 Experimental Setting
Generative Model. For embedding model training, we fine-tune tune the model as defined in Equation For synthesis model training, only update image projector weights, image attention weights, and LoRA weights. We adopt classifier free guidance (CFG) [32] with a strength of 5.0, and randomly apply conditional dropout to the embedding control and text control, with 5% chance drop the control from image, text or both, respectively. More details are provided in the Appendix.
Synthesis for Analysis. We generate synthetic images at 512 resolution and downsample them to 256 for storage, following diffusers [78] parameter settings. The synthetic dataset pool consists of approximately K=5 times the number of original training samples. For example, for category with n real samples, we generate 5n text prompts from LLM, and then produce the same amount of synthetic images of the same category. Under few-shot and long-tail settings, ensure more balanced class distribution by generating at least 250 (= 50 x K) synthetic samples per class.
We use CLIP pre-trained RN50 and ViT-B/16 with input resolution 224x224 as our backbone models, each topped with a 3-layer MLP projection head plus batch normalization. We apply Rugmenttent [16] while holding out Mixup and CutMix for comparison. In few-shot settings, we freeze the backbone and train only the projection head for 10% of the standard epoch count. Additional training parameters are detailed in the Appendix.
Classifier.

---

Model
Method
IN-LT
Head
Mid
Tail
iNat
Head
Mid
Tail
Vanilla
69,49
83.12
64.96
39.82
65.52
76.54
68.31
59.25
CutMix 3X
72.69
84.34
68. 18
42.73
66.47
77.49
69.26
60.20
Mixup
70.73
84.38
66.22
41.28
67.59
77.61
70.38
61.32
VT1-
Da- Pusion
76
71.96
83.61
70.45
44.63
66.61
77.51
69.35
60.27
Ours
79.17
83.77
78.29
69.93
74.11
77.36
74.12
73.20
Vanilla LWS
72.28
79.78
69.83
56.77
71.97
74.23
73:45
69.83
Ours + LWS
80.58
81.97
79.73
74.51
78.54
75.58
78.84
79.03
Table 3: Long-tail classification Top-1 accuracy (%). Results are reported separately for head, mid, and tail categories. Our approach outperforms existing data augmentation methods by 6.5%. especially for tlis-classes. The performance of our method can be further improved through class re- balancing method such as LWS.
Model
Method
L.L
W.W
L.W
W.L
Avg.
Vanilla
43.68
36.24
43.39
35.92
39.81
Real Real-GGidannce
44.62
37.73
45.29
35.50
40.79
VIT-B/16
Da-Fusion 76
49.63
43.13
49.95
39.21
45.48
Diff-Mix 83
43.57
37.09
42.56
37.61
40.21
Ours
54.86
45,53
56.51
47.74
51.16
Table 4:
OOD classification Top-1 accuracy (%) on the Waterbird dataset. Our method is robust to unseen domain shifts by reducing spurious relations between background and foreground elements.
Datasets. We conduct experiments on widely used Fine-Grained Visual Classification (FGVC)
datasets: ImageNer- 1K
[17 (IN), iNaturalis22018 (77 (iNat), Flower102 10 [56], Aircraft100 [5])
CUB200-2011 (79. Cars102
4 StanfordDogs 120 41 and Food101 9 Here, we clarify that we
trained individual generation model for each dataset.
Peer Methods. We implement Mixup (94] and CutMix [93 with probabilities of 0.5 and 0.3 respec- tively. We also evaluate against GDA methods such as RRal-GGuidance [28]. DA-Fusion (76), and Diff-Mix [83], following their official implementations. We also compare with long-tail recognition methods such as LWS [39] in the main text and others [100] 81 34
the Appendix.
4.2 Performance of Generative Data Augmentation
In this section, we evaluate the effectiveness of our approach across conventional, few-shot, long-tail, and cut-of-distributie scenarios. The reported results are the averaged over three independent trials with different random seeds.
Conventional Classification Performance. In Table 1 we perform end-to-end training and evaluate the performance of classifiers trained with real and synthetic data compared to baseline methods on 6 FGVC datasets. Our method outperforms existing GDA techniques, achieving average improvement of 0.73% on RN50 and 0.55% on ViT-B/16.
Few-Shot Learning Performance. We adopt an N-way 10-shot setting over three episodes, where N the number of classes. For each class, we randomly draw 10 samples from the training set, resulting in a total of 10N training data. We use this same limited dataset both for fine-tuning the generation model at 15% of the epochs compared conventional setting and training the downstream classifier. Using fine-tuned generation model, we generate 250 synthetic images per category. During classifier training, we freeze the CLIP backbone and fine-tune classification head for 15 to 50 epochs, depending on N. As shown in Table our method achieves the highest average performance across all datasets, achieving an improvement over existing methods by 1.85% on ViT-B/16.
Long-Tai Tail Classification Performance. We follow the work of [49 59 to conduct experiments on IN-LT and iNat. Our generation model is trained on the same dataset as the classification model. We generate at least 250 images for all categories for better class balancing. As shown in Table 3 our method achieves a remarkable performance improvement of 6.48% and 6.52% overall and 25.30% and 11 1.88% on tail categories for IN-LT and iNat, respectively. This improvement is more significant than in the few-shot setting. We hypothesize that this because the synthesis model benefits from a larger number of training samples in the long-tail scenario, allowing it to better understand the relationships across categories and learning to generalize to capture salient concept for tail categories.
As result, downstream classifier Iearns more robust representations for the tail class by training
7

---

Dutuset
Synthesis Model
Dreamnimm
Vendit
CLIPT
Gen. Tops
Real-CCuidance
28
0.4709
17.22
0.3396
88.27%
Da-Pusion 76
0.4531
13.26
0.2801
76.93%
IN
Dift-Mix
84
0.4399
15.81
N/A
91.27%
Ours
0.3930
16.76
0.3119
98.84%
Real- Real-GGidaance
28
0.5370
34.27
0.3302
31.09%
Da-Fusion 76
0,5426
28.
0.2387
27.60%
iNat
Diff-Mix 83
0.5224
31.57
N/A
45.34%
Ours
0,4293
34.48
0.3106
87.48 %
Table 5: Comparison of quality of the generated image from different DA methods. We use DreamSim to measure the foreground subject consistency and Vendi embedding score for measuring generated image diversity. We further use Clip measure the consistency between input text prompt and the output image and evaluate the accuracy external classifier on the generated images (Gen. Top5). Geal-Guid uses T2I generation models and therefore has the best text-image consistency. Our method achieves the overall best trade-off between salient concept consistency and diversity, particularly for fine-grained dataset such as iNat.
Data
Model
DreamSimi,
CLIPT
Gen. Top5
Cls. Top1
Ours w/o SCA (CE)
0.4626
0.2998
79.48%
66.24%
Ours w/o SDXL(+SDI5)
0.4587
0.2785
75.79%
7188%
Ours w/o LoRA
0,4506
0,286
80.71%
68.86%
iNat
Ours w/o (dapter(+MLP)
0.4335
0,3048
87.17%
73.43%
Ours w/o Class Cond. Ours
0.4321
0,3092
86.44%
73.77%
0.4293
0.3106
87.48%
74.01%
Table 6: Ablation study of the contribution of individual components in our synthesis pipeline on the iNat dataset. Here, Gen. Top5 measures the accuracy of external classifier on the generated images, and Cls. Top1 measures the accuracy of downstream classifier trained on the generated data. We selectively removed or replaced the proposed components to demonstrate their respective contributions. particular, replacing the SCA embedding with classifier emhedding significantly degrades performance.
more faithful and diversified data. We will discuss more about the generalization capability of the synthesis model in the next section.
We further apply learnable weight scaling (LWS) 33 that fine-tunes additional weight scaling factors for 40 epochs to improve the performance. At this stage, the model is trained on balanced dataset consists of class-balane balance sampling from the real dataset with probability (1 P)% and sitance-ballncee sampling from the synthetic dataset with probability P%. The performance can be further improved by 1.41% and 4.43% for IN and iNat, respectively. We also conduct experiments on other re- balancing and calibration methods in the long-tail recognition literature [100] 81 34). We find that LWS works the best for our settings and we will present the results for others in the Appendix.
Performance under OOD Setting. We conduct out out-of-distin distriibiittion (OOD) evaluation using the Waterbird dataset (66), which is created by superimposing foregrounds from the CUB dataset onto backgrounds from the Places dataset [101]. The results of our OOD evaluation are presented in Table where "L" and "W" denote land and water, respectively. For example, "L.W" means a landbird placed against water background. We train the model following N-way 5-shot scenario over three episodes using the CUB dataset, augmented with synthetic data generated through our GDA framework. Our approach is well-suited for this challenge, outperforming peer approaches by 5.68%, as our inference data construction in Section 3.3 explicitly incorporates diverse background variations, allowing classifier to mitigate spurious correlations with background elements.
4.3 Ablation Studies
Performance of Personalized Image Generation. In this section, we directly assess the diversity and faithfulness of our method's s generated data and compare it with peer methods. For concept preservation, we use external expert classifiers from timm 85) leaderboard that achieve best recogni- tion accuracy on target image domains. We report classification performance using Top-5 accuracy on synthetic images. To evaluate text-image aligmment, we compute the cosine similarity between feature embeddings extracted from the CLIP-B/16 model's text and image encoders for input text prompts and generated images, respectively. We additionally use DreumSim 23 for measuring fore- ground subject consistency and Vendi (22 embedding score for measuring generated image diversity.
The evaluation is conducted image-text prompt pairs explained in Section 3.3 used for training
8

---

100
95
Performance
90
65
80
70
Airmafts
Degs
65
INat
Flowers
Food
CUB
Cars
P
K
Figure 4: Impact of synthetic-to-real data ratio K and synthetic sampling probability P on Top-1 classification accuracy. Performance improves when a balanced proportion of synthetic data is used, but degrades when synthetic samples dominate. Increasing P improves datasct diversity, but the benefits diminish as P grows.
downstream classifiers, without applying any filtering. We present performance results in Table 5 The dataset generated by our method is diversified, achieving decent vendi and text consistency scores, and faithful, achieving best dreamsim scores and evaluation classifier accuracy. The advantage significant for more fine-grained dataset like iNat, where we outperform the second-best model by 42. 14% in Top-5 accuracy.
We intentionally exclude metries such as CLIP image embedding consistency and widely used measures like FID [3]] and KID [8] scores that favor identity mapping, which do not align with our objective of generating images that are meaningfully different from the input. For example, FID is minimized when comparing two identical datasets because it quantifies the distance between two distributions.
Effectiveness of Components. To assess the effectiveness of the SCA embedding, we train a classifier using a standard cross-entropy (CE) loss for image prompt feature extraction. As shown in Table this results in performance drop across all metrics, meaning that conventional CE-based embeddings are inadequate for preserving key concepts and generating diversified augmentations. Additionally, we examine the impact of key architectural choices in our generation framework. Specifically, we replace the SDXL backbone with Stable Diffusion 1.5 (SD15), substitute the adapter with a naive MLP, remove LoRA, and replace the image classifier-free null guidance with zero vector. Across these ablations, we always observe performance drop across all metries. This shows that each part of our architectural design is integral in improving the overall performance.
Synthetic Ratios and Sampling Probabilities. We conduct ablation study two key factors affecting downstream classification performance: the synthetic-to-real data ratio K and the probability of sampling synthetic data P during training, both defined in Section 3.3 In Figure we analyze the Top-1 accuracy of the downstream classifier across different values of K = {0. 0.2, 0.5, 0.75, 1.0} and P {0, 1, 2, 3, 4, 5). Our default settings use K 0.2 for IN and iNat and K 0.5 for other FGVC datasets, with P = 5. The results indicate that performance improves when K is within the range of 0.2 0.5, but degrades when K becomes excessively large. Incorporating synthetic data improves model generalization, but they may also introduce distributional shifts that negatively impact classification performance. Similarly, the sampling probability controls the diversity of the training set. Performance increases as P grows but stabilizes beyond P= 3.
Computational Cost Analysis. A practical consideration for any data augmentation method is its computational overhead. Our approach involves two training stages: (1) training the SCA embedding model, and (2) fine-tuning the synthesis model using SCA embeddings. We provide a detailed analysis of the computational requirements for both stages. Training the SCA embedding model requires approximately 20% of the time needed to fine-tune the synthesis model. Table reports wall-clock training times on an 8 Xx.A A100 GPU setup across datasets of varying scales. For typical FGVC datasets with approximately 10K images, the total training time is 20-30 minutes. For larger-scale datasets such as iNat2018 (440K images) and ImageNet- 1K (1.3M images), training requires 19 and 25 hours, respectively. For the largest dataset we experimented with, iNat2021 containing 2.7M images, training takes approximately 3 days. We argue that this computational cost is practical for most application scenarios, especially in domains where curating high-quality in-domain data prohibitively expensive or time-consuming. Once trained, the synthesis model can generate unlimited augmented samples

---

Dataset
FGVC
iNa22018
ImageNet-IK
iNat2021
Dataset Size
10K
440K
1.3M
2.7M
Training Time
20-30 mins
19 hours
25 hours
3 days
Table Training time for method reasonably with dataset size, ranging from 20-30 minutes for typical FGVC datasets to approximately days for the largest dataset (iNat2021 with 2.7M images).
datasets of different scales 18xAI00 GPUs). Training time scales
efficiently during downstream classifier training, providing consistent performance improvements over baseline GDA methods with a one-time training investment.
Conclusion
In this paper, introduce approach to generative data augmentation (GDA) using person- alized image generation framework with a salient concept-aware (SCA) image embedding model, mitigating the fidelity-diversity tradeoff.. SCA effectively captures key information from image prompts while preserving the semantics specified by text prompts, maintaining intuitive alignment across both modalities. Our method generates training data with focused diversity that significantly improve the robustness of the model, especially under long-tail settings. Experiments across eight fine-grained datasets show that our method outperforms existing GDA methods, particularly under long-tail and cutoof-distribution settings. Through ablation studies, we compare the quality and diver- sity of our generated images with peer GDA methods, validate the effectiveness of each component in our synthesis pipeline, and studied the impact factors on the downstream classification performance. Our method is proved be a promising approach for generative data augmentation.
References
[1] Xiang Jiankang Deng, Kaicheng Yang, Jaiwei Li, Ziyong Feng, Jia Guo, Jing Yang. and Tongliang Liu. Unicom: Universal and compact representation learning for image retrieval. arXiv:230k 05884, 2023.
[2] AI Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 2024.
[3] Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain-agnostic tuning-encoder for fast personalization of text-to-image models. In SIGGRAPH Asia, pages 1- 10, 2023.
[4] Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. Break-a-scene: Extracting multiple concepts from single image. In SIGGRAPH Asia, pages 12, 2023.
[5] Shekoofeh Azizi. Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David Fleet. Synthetic data from diffusion models improves imagenet classification. arXiv:2304.088666 2023.
[6] Yogesh Balaji. Seungjun Nah, Xun Huang. Arash Vahdat, Jiaming Song. Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli I Laine, al. ediff-i: Text-to-image diffusion models with ensemble expert denoisers, arXiv:22 3244, 2022.
[7] Hritik Bansal and Aditya Grover. Leaving reality to imagination: Robust classification via generated datasets. arXiv:2302. 200.:22322.00033 2023.
[8] Mikolaj Birikowsii, Danica Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv:1801 000110000001111 2018.
[9] Lukas Bossard, Matthieu Guillaumin, and Lue Van Gool. Food-101 10mmming discriminative components with random forests. In ECCV, 2014.
[10] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpixppix: Learning to follow image editing instructions. CVPR, 2023.
[11] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
[12] Ruoxin Chen, Zhe Wang, Ke- Yue Zhang, Shuang Wu, Jiamu Sun, Shouli Wang, Taiping Yao, and Shouhong Ding. Decoupled data augmentation for improving image classification. arXiv:2411.02592 2024.
10

---

[13] Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W Cohen. Re-imagen: Retrieval-augmented lext-to-image generator. arXiv:2209. 14491,
[14] Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming- Min Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. NeurIPS, 2024.
[15] Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. 23X.v:3307.0988/ 2023.
[16] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with reduced search space. In CVPRW, 2020.
[17] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: hierarchical image database. In CVPR, 2009.
A large-scale
[18] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In CVPR, 2019.
[19] Ming Ding, Zhuoyi Yang. Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, al. Cogview: Mastering exx-tt--mmaee generation via transformers. NeurIPS. 2021.
[20] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. arXiv:2433.03208 2024.
[21] Lijie Fan, Kaifeng Chen, Dilip Krishnan, Dina Katabi, Phillip Isola, and Yonglong Tian. Scaling laws of synthetic images for model training... for now. In CVPR, 2024.
[22] Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning, arXiv:2210.02411 2022.
[23] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. arXiv:2306.099 093444 2023.
[24] Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene, Scene-based text-to-image generation with human priors. In ECCV, 2022.
[25] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image worth one word: Personalizing text-to-image generation using textual inversion. arXiv:2208.01668, 2022.
[26] Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-On Encoder- based domain tuning for fast personalization of text-to-image models. TOG, 42(4):1-133 2023.
[27] Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdifff: Compact parameter space for diffusion fine-tuning In ICCV, 2023.
[28] Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenging Zhang, Philip Torr, Song Bai, and Xiaojuan Qi. Is synthetic data from generative models ready for image recognition? arXiv:2210.07574 2022.
[29] Zecheng He, Bo Sun, Felix Jucfei-Xuu Xu, Haoyu Ma, Ankit Ramchandani, Vincent Cheung, Siddharth Shah, Aumol Kalia, Harihar Subramanyam, Alireza Zarcian, al. Imagine yourself: Tuning-free personalized image generation. arXiv:2409. 13346, 2024.
[30] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfrr Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. 1:XXi:2208.01622 2022.
[31] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. NeuriPS, 2017.
[32] Jonathan Ho and Tim Salimans. free diffusion guidance. arXiv:2207.12598 2022.
[33] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistic models. NewrIPS, 2020.
[34] Youngkyu Hong. Seungju Han, Kwanghee Choi. Seokjun Seo, Beomsu Kim, and Buru Chang. Disentan- gling label distribution for long-tailes tailed visual recognition. In CVPR, 2021.
11

---

[35] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. vrXv:22006009655, 2021.
[36] Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou, Composer: Creative and controllable image synthesis with composable conditions. arXiv:2302.0977 2023.
[37] Khawar Islam, Muhammad Zaigham Zaheer, Arif Mahmood, and Karthik Nandakumar. Diffusemix: Label-preserving data augmentation with diffusion models. In CVPR, 2024.
[38] Xuhui Jia, Yang Zhao, Kelvin CK Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXiv preprint arXiv:2304.02644 2023.
[39] Bingyi Kang. Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. 191.0:1910.09217, 2019.
[40] Bahjat Kawar. Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, 2023.
[41] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine- grained image categorization: Stanford dogs. In CVPR Workshop. 2011.
[42] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg. Wan- Yen Lo, et al. Segment anything. In /CCV, 2023,
[43] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV Workshop, 2013.
[44] Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of rext-to-image diffusion. In CVPR, 2023.
[45] Bohan Li, Xiao Xu, Xinghao Wang, Yutai Hou, Yunlong Feng, Feng Wang. Xuanliang Zhang, Qingfu Zhu, and Wanxiang Che. Semantic-guider generative image augmentation method with diffusson models for image classification. In AAAI, 2024.
[46] Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. NeurlPS, 2024.
[47] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In ICML, 2023.
[48] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang. Jianfeng Gao, Chunyuan Li, and Yong Jac Lee. Gligen: Open-set grounded text-to-image generation. In CVPR, 2023.
[49] Ziwei Liu. Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large-scale long-tailes recognition an open world. In CVPR, 2019.
[50] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv:1711.05100,,22 2017.
[51] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine Fine-grained visual classification of aircraft. arxivv1306.5155 2013.
[52] Chenlin Meng. Yutong He. Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. ICLR, 2022.
[53) Eyal Michaeli and Ohad Fried. Advancing fine-grained classification by structure and subject preserving augmentation. arXiv:2406. 14551, 2024.
[54] Chong Mou, Xintao Wang, Liangbin Xie. Yanze Wu, Jian Zhang, Zhongang Qi. and Ying Shan. T2i- adapter: Learning adapters to dig more controllable ability for text-to-image difffusion models. In AAAI, 2024.
[55] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen, Glide: Towards photorealistic image generation and editing with lext-guided diffusion models. 1rXiv:21/2.10744 2021.
[56] M-E Nilsback and Andrew Zisserman. A visual vocabulary for flower classification. In CVPR, 2006.
12

---

[57] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, al. Dinov2: Learning robust visual features without supervision. arXiv:2304.07193, 2023.
[58] Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, Kfr Aberman, et al. Moa: Mixture-of atxtutioof-attention for subject-context disentanglement personalized image generation. arXiv:2404. (1565, 2024.
[59] Sculki Park, Youngkyu Hong, Byeongho Hco, Sangdoo Yun, and Jin Young Choi. The majority can help the minority: Context-rich minority oversampling for long-tailed classification. In CVPR, 2022.
[60] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. 2rXii:2307.01952. 2023.
[61] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et Learning transferable visual models from natural language supervision. In ICML, 2021.
[62] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen, Hicrarchical text-conditional image generation with clip latents. arXiv:2204.06125. 2022.
[63] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
[64] Olaf Ronneberger, Philipp Fischer. and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
[65] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yacl Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023.
[66] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. 1rXiv:1911.08731, 2019.
[67] Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: mmage-to-image diffusion models. In SIGGRAPH, 2022.
[68] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorcalistic text-to- image difffusion models with deep language understanding. NewrlPS, 2022.
[69] Mert Bülent Sanyıldız, Karteek Alahari, Diane Larlus, and Yannis Kalantidis. Fake till you make it: Learning transferable representations from synthetic imagenet clones. CVPR, 2023.
[70] Jing Shi, Wei Xiong. Zhe Lin, and Hyun Joon Jung. Instantbooth: Personalized text-to-image generation without test-time finetuning. arXiv:2304.0341 2023.
[71] Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang. Jarred Bar- ber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image generation in any style. 3.iv:22306.00988. 2023.
[72] Jiaming Song, Chenlin Meng, and Stefano Ermon. arXiv:2010.02502, 2020.
Denoising diffuusio implicit models.
[73] Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. Stablerep: Synthetic images from text-to-image models make strong visual representation learners. NewrIPS, 2023.
[74] Yonglong Tian, Lijic Fan, Kaifeng Chen, Dina Katabi, Dilip Krishnan, and Phillip Isola. Leamingg vision from models rivals learning vision from data. In CVPR, 2024.
[75] Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. Stablerep: Synthetic images from text-to-image models make strong visual representation learners. NewrlPS, 2024.
[76] Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov. Effective data augmentation with diffiusion models. ICLR, 2024.
[77] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR, 2018.
13

---

[78] Patrick von Platen, Suraj Patil. Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul. Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-ard diffusion models, 2022.
[79] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
[80] Haofan Wang, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style-preserving in text-lo-image generation. AXXY:22404.02733. 2024.
[81] Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X Yu. Long-tailed recognition by routing diverse distribution-awarc experts. ar arXiv2200000/800 2020.
[82] Yanghao Wang and Long Chen. Improving diffusion-basee based data augmentation with inversion spherical interpolation. arXiv:2408, 16266, 2024.
[83] Zhicai Wang, Longhui Wei, Tan Wang, Heyu Chen, Yanbin Hao, Xiang Wang, Xiangnan He, and Qi Tian. Enhance image classification via inter-class image mixup with diffusion model. In CVPR, 2024.
[84] Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text- totttoo-mmae generation. In ICCV, 2023.
[85] Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training procedure in timm. arXiv:2110.00470 2021.
[86] Guangxuan Xiao, Tianwei Yin, William T Freeman, Frédo Durand, and Song Han. Fastcomposer: Tureng-free multi-subject image generation with localized attention. arXiv:2300. 10441, 2023.
[87] Zeyue Xue, Guanglu Song. Qiushan Guo, Boxiao Liu, Zhuofan Zong, Yu Liu, and Ping Luo. Raphael: Text-to-iman image generation via large mixture of diffusion paths. NeurlPS, 2024.
[88] Han Yang. Chuanguang Yang. Qiuli Wang, Zhulin An, Weilun Feng, Libo Huang, and Yongjun Xu. Multi-party collaborative attention control for image customization. arXiv preprini arXiv:2505.01428, 2025.
[89] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arxiv:2308.067. 006701 2023.
[90] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, al. Scaling autoregressive models for content-rich lext-to-image generation. arXiv:2206. 10789, 2022.
[91] Zhuoran Yu, Chenchen Zhu, Sean Culatana, Raghuraman Krishnamoorthi, Fanyi Xiao, and Yong Jac Lee. Diversify, don't fine-tune: Scaling up visual recognition training with synthetic images, arXiv.2312.02255, 2023.
[92] Jianhao Yuan, Francesco Pinto, Adam Davies, and Philip Torr. Not just pretty pictures: Toward interven- tional data augmentation using text-to-image generators. arXiv:22/2. 1/37, 2022.
[93] Sangdoo Yun, Dongyoon Han, Scong Joon Oh, Sanghyuk Chun, Junsuk Choc, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, 2019.
[94] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. 1rXiv:1710.094122 2017.
[95] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
[96] Xulu Zhang, Xiao- Xiao-Yong Wei. Wengyu Zhang, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, and Qing Li. A survey on personalized content synthesis with diffusion models. arXiv:2405.05534 2024.
[97] Qihao Zhao, Yalun Dai, Hao Li, Wei Hu, Fan Zhang, and Jun Liu. Ltge: Long-tail recognition via leveraging llms-driven generated content. In CVPR, 2024.
[98] Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. NewrIPS, 2023.
[99] Chenyu Zheng, Guoqiang Wu, and Chongxuan Li. Toward understanding generative data augmentation. NeurIPS, 2023.
14

---

[100] Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long-tailed recognition. In CVPR, 2021.
[101] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. PAMI, 2017.
[102] Yongchao Zhou, Hshmat Sahak, and Jimmy Ba. Training on thin air: Improve image classification with generated data. arXiv:2305.15316, 2023.
[103] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny, Minignt-4: Enhancing vision-language understanding with advanced large language models. arXiv:2304. 10592, 2023.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction of this paper accurately reflect its contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA]
16

---

Justification: N/A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Experiment details are provided in this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Legal constraints.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pbllc/guudeeee http://oeeubiiissioooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
18

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
All the training and test details specified.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Our results are averaged over trials.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide them in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
19

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The work has no social impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
20

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We properly cited all assets being used.
Guidelines:
The answer NA that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
21

---

Justification: N/A
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy http:s/s/meerppppeec//oooerreees for what should or should not be described.
22

---

A Model Preliminaries
Synthesis Model Preliminary. Diffusion models [33] can generate high-fidelity images from noise in an iterative manner, which is often accelerated with fast samplers like DDIM [72]. For conditional diffusion models, techniques like classifier-free guidance [32] are employed balance image fidelity and sample diversity, realized by randomly dropping the condition during training. In this study, we adopt Stable Diffusion XL (SDXL) [60 as our base architecture. As latent diffusion model [63] SDXL built upon UNet [64] with attention layers, conditioned on text features extracted from two frozen CLIP text encoders.
Synthesis Model Size Statisties. SDXL. is our baseline text-to-imaae model with 3.856 billion parameters. SDXL.-UNet and SIXXI-VVEE are the U-Net and VAE components with 2.955 billion and 84 million parameters, respectively. original SDXL paper with total of 818 million parameters. Adapter is our proposed module for fusing text and image representations, with 73 million parameters. LoRA is a technique for efficiently adapting new concept knowledge into pre-trained models, with a total of 387 million parameters. The Image Embedder is our module for embedding image representations, with 87 million parameters. Our adapter plus LoRA approach adds a total of 460 million parameters to the base SDXL model.
TeXX--TextEneoder are the two text encoders proposed in the
Classifier. For all experiments, we employed the official CLIP architecture with RN50 and ViT-B/16 backbones.
B More Training Details
SCA Training. Our SCA embedding model is developed from a pre-trained DINOv2-Baa back- bone [57], connecting with head component with two linear layers containing batch normalization, and one fully connected layer. We fine-tune the whole SCA model with equals to 32, e equals to 0.35. Following (1) set the training epoch as 32, learning rate as 4e-5, and batch size as 128, train with 8x A10 GPUs using Adam W (50) optimizer. After training, we normalize the embedding from the backbone and feed it into our diffusion model.
SDXL Training. Diffusion model backbone SDXL 60 implemented in Diffusers [78 reposi- tory, model originally trained on high-resolution images of size 1024. To optimize memory usage and accomodate lower resolution training data, we centercrop the image followed by resizing to size of 512, allowing an effective batch size of 64 when distributed across 8x A100 GPUs. We use Blip2 [47] and MiniGPT-4 [003] to generate textual prompts for each training image. During the training process, encourage diversity and improve the model's robustness in understanding prompts, we randomize the training prompts by splitting the generated prompt paragraphs into short sentences and key terms, which are then randomly sampled and recombined form the final training prompts. Furthermore, we adopt classifier-free guidance (32), randomly applying conditional dropout to the embedding control and text control, with 5% chance drop the control from image, text or both, respectively. We also apply data augmentation techniques such as horizontal flipping and color jittering. We use the Adam W (50) optimizer with a constant learning rate of 1e-5, where we update only the image projector weights, image attention weights, and LoRA weights. We train the adapted model for 10 epochs across all datasets. We observe that smaller datasets converge faster, while larger datasets converge slower. For example, it takes approximately 3 days to converge with 8x A 100 GPUs for iNaturalist
when training on set of about 2.7 million images.
The training objective is to reconstruct the image prompt given the caption prompt. Taking Ima- g (INIK) as an example, for training, the image prompts were sourced from the training split, and the corresponding text prompts were generated by the publicly available Blip and MiniGPT models. We forward pass the image æ through SCA and and text through language encoder obtain respective embeddings SSCA
"mage. Ziexr which are taken by the synthesis model as inputs, and the image being the ground truth target for the output. Namely, training triplet is consisting of (zimagee Riest x). Since the c loss in the training phase do not require us to generate completed images, we only have the generated images available during the evaluation phase, where the image prompt and text prompt are the inputs, and generated image is the output.
Classifier Training. We primarily followed the training protocol established in Azizi et al. (5) with modifications tailored to our specific requirements. For ResNet50, we train ImageNet- 1K and iNaturalist 2018 for 300 and 450 epochs respectively, and for smaller datasets we use 100 epochs;

---

Model
Method
IN-LT
Head
Mid
Tail
iNat
Head
Mid
Tail
CE
79.17
83.77
78.29
69.93
74.11
77.36
74.12
73.20
cRT
39
80.41
82.56
79.48
72.79
77.14
76.21
76.86
77.67
LWS 39
80.58
81.97
79.73
74.51
78.54
75.58
78.84
79.03
ViT-B/16
LAS [00]
79.96
82.1
78.86
72.84
76.35
75.07
76.23
76.81
RIDE-3
[1
78.95
79.85
78.41
73.27
75.05
72.66
75.35
75.37
PC Softmax
34
79.98
81.52
79.04
73.96
76.62
74.85
76.09
77.63
Table & Comparisons among long-tail recognition methods, reported in classification Top-1 accuracy (%). The performance of the classifier can he further improved by fine-tuning the header with re-halancing or calibration approaches.
ViT-B/16 requires 60% more epochs across all datasets. The learning rate decays by a factor of 10 at 50% and 75% of total epochs. More precisely, for example, for large-scale datasets like ImageNet. we consider the following settings: for the ResNet50 model, training goes for 300 epochs with an input size of 224 x 224 and large batch size 8192. We use SGD optimizer with learning rate of 3.6, a cosine decay schedule and weight decay of 1 x 10-4 There's 5-epoch warmup phase. On the other hand, the ViT-B/16 model is trained for 480 epochs with the same input size but smaller batch size of 1024, using the Adam W optimizer with lower initial learning rate of 1 x 10-3 also decaying via cosine schedule but with a higher weight decay of 0.05. Its warmup period is longer at 10 epochs, and it additionally uses drop path rate of 0.2. Both models apply standard random cropping of 224 x 224 pixels, a horizontal flip probability 0.1, and RandAugment set to for data augmentation. For FGVC datasets, we reduced the training duration to one-third the epochs. We further adjusted the hyperparameters for few-shot learning and OOD detection tasks to optimize performance.
0.5, label smoothing with a factor of
C Long-Tail Recognition Methods
Our method exhibits compelling performance under the long-tail setting, We further apply long-tail recognition methods upon our generated data investigate potential performance improvements. Using the representation from the CE pre-trained model, we fine-tune for an additional 20-50 epochs during the second stage training on the classifier header (or experts for RIDE). For re-balancing approaches, we implement class balance strategies for real and synthetic data separately, as the latter more balanced by construction. More precisely, the stage-two balanced dataset consists class- sampling
balanced sampling from the real dataset with probability (1 P)% and from the synthetic dataset with probability P%. For calibration approaches, we employ heuristic approach that recalibrates the number of data points to be weighted average of real and synthetic data: n = anaaal + (ffke where a = O.S. The results presented in demonstrtt that our method achieves superior performance when combined with additional re- re-balancing calibration methods. In particular, learnable weight scaling (LWS) achieves the best results compared to other methods for both datasets, outperforming the baseline by 1.41% and 4.43%.
D Broader Applicability Beyond Fine-Grained Classification
Although our main experiments focus on fine-graines visual classification, this choice is motivated by practical challenges in niche domains where expert models must be trained on comain--speciff datasets that are often small or class-imbalann imbalanced. GDA is particularly valuable in these settings, as it can generate faithful and diverse samples by leveraging world priors learned from synthesis models, thereby improving robustness of downstream classifiers. We note that our current synthesis pipeline, built on SDXL, is optimized for object-centric image generation and may not be directly suitable for complex vision tasks such as text-rich VQA or GraphQA. Nevertheless, we demonstrate the broader applicability of our method through two additional experimental settings: abstract scene understanding and multi-modal visual classification.
D.1 Abstract Scene Understanding: Places365
To evaluate performance beyond object-centric datasets, we conduct few-shot experiments on Places365 [101]. large-scale dataset for scene recognition. We follow the same N-way 10-shot

---

Method
Vanilla
CutMix
Mixup
Da-Fusion
Diff-Mix
Ours
Top-1
35.32
36.23
35.53
36.57
37.53
39.06
Top-5
64.15
65.62
64.47
65.88
66.91
68.69
Table 9: Classification accuracy (%) using 10-shot training samples on Places365 validation Our method achieves consistent improvements over baseline augmentation methods.
Method
CUB
Aircraft
Flowers
Cars
Dogs
Food
Average
Zero-shot
79.44
58.78
82.91
87.58
78.48
90.31
79.58
Fine-tuned (Diff-Mix)
89.37
66.92
94.71
95.30
87.14
83.23
86. 36.11
Fine-tuned (Ours)
90.49
79.31
95.30
96.57
87.33
91.53
90.09
Table 10: Classification accuracy (%) under N-way 10-shot setting with Qwen2.5- backbone. Our method demonstrates consistent improvements both zero-shot and Diff-Mix augmented fine-ture fine-tuning.
experimental protocol described in Table 2 of the main paper, with evaluation performed on the validation set. As shown in Table 9 method outperforms the strongest baseline (Diff-Mix) by 1.53% and 1.78% Top-1 and Top-5 accuracy, respectively. This demonstrates that our approach generalizes effectively abstract scene understanding tasks beyond fine-grained object classification.
D.2 Multi-Modal Visual Classification
Classification remains important task for modern multi-modal models, which often require fine- tuning on domain-sppp -ppecin datasets to achieve production-leve performance. To demonstrate the effectiveness of generation method in this context, explore multi-modal visual classification by performing supervised fine-tuning on a multi-modal large language model using FGVC datasets.
We use Qwen2.5- V VL.-BB as our backhone model. During fine-tuning, we tune only the vision projector and vision encoder while keeping the L.M frozen. The classification problem is reformulated as a 26-choice multiple-choice QA task, where the ground-trutt label is always included among the choices. We use the following prompt template:
"You are an expert classifier. Based on the visual evidence in the image, select the most appropriate category. Please choose ONE of the following: A. beef tartare B. preme_brutee Z. lacos. Please respond ONLY with your answer wrapped in this format: <answer> LETTER </answer>."
Table 10 presents classification accuracy under the N-way 10-shot setting across six FGVC datasets. Zero-shot performance is generally unsatisfactory, particularly on specialized domains like Aircraft recognition. When fine-tuning with augmented data generated by our method, we achieve substantial improvements over Diff-Mix, with an average gain of 3.98% across all datasets.
E Complete Image Quality Evaluation
In Section 4.3 of the main paper, presented image quality evaluation ImageNet and iNaturalist, the two largest and most challenging datasets in our benchmark. For completeness, Table [I presents the full evaluation across all six remaining FGVC datasets. Our method consistently achieves the best performance across all metrics and datasets. Notably, our approach maintains superior diversity (Vendi Score) and simultaneously improves fidelity (lower DreamSim) and generation accuracy. This demonstrates that the didelity trade-off is effectively balanced by our SCA embeddings across different visual domains, from fine-grained flowers and birds aircraft and food categories.
F Comparison with Off-the-Shelf Diffusion Adapters
To contextualize our approach within the broader landscape image-conditioned diffusion models, we compare our method against several state-of-the-ar off-the-shelf adapters: IP-Adapter [89]. ELITE (84, E4T (26), and FastComposer [86]. These methods provide pre-trained image encoders and adapter modules that can be directly applied to domains without task-specific fine-tuning.
3

---

Dataset
Method
DreamSim
Vendit
CLIP-TY
Gen. Top-1
Gen. Top-5
Real-Guidanee
0.3307
15.16
0.3236
87.31 %
95.30%
DA-Fusion
0.3260
13.06
0.2795
88.24%
96.22%
Flowers
Diff-Mix
0.3195
14.20
N/A
91.83%
99.00%
SCA (Ours)
0.3051
15.31
0.3089
95.27 %
99.88% %
-aa--Guidence
0,4382
8.01
0.3627
54.27%
82.93%
DA AA-Fusion
0,4263
7.59
0.3169
60.85%
85.60%
Cars
Diff-Mix
0,4035
7.72
N/A
85,46%
98.69%
SCA (Ours)
0.3880
7.96
0.3411
86.60 %
99.07% %
Real-Guidance
0.4676
7.59
0.3411
25.67%
65.99%
DA-Fusion
0.4549
6.54
0,2974
33.58%
71.35%
Aircraft
Diff-Mix
0,4230
7.04
N/A
49.46%
81.96%
SCA (Ours)
0,4001
7.61
0.3167
60.17% %
92.95 %
Real-Guidance
0.4584
11.91
0.3479
63.03%
93.75%
DA-Fusion
0,4624
11.81
0.3033
62.14%
90.97%
CUB
Diff-Mix
0.4379
11.49
N/A
69.53%
96.49%
SCA (Ours)
0,4092
11.89
0.3283
73.66 %
97.64%
Geal-Guidance
0.4833
21.44
0.3413
87.71%
99.03%
DA AA-Fusion
0.4796
19.46
0,2974
87.56%
98.4 28.41%
Food-101
Diff-Mix
0,4513
20.13
N/A
90.16%
99.22%
SCA (Ours)
0,4374
21.31
0.3289
92.48 %
99.84% %
Real- Real-Guidance
0,4731
37.92
0.3668
79.38%
96.09%
DA-Fusion
0,4569
32.07
0.3011
79.74%
97.27%
Dogs
Diff-Mix
0.4274
34.73
N/A
81.24%
97.40%
SCA (Ours)
0.4105
37.99
0.3439
83.78% %
98.26
Table 11: Complete image quality evaluation across all FGVC datasets. report perceptual similarity (DreamSim 4), sample diversity (Vendi Score 7), text-image alignment (CLIP-T t), and generation accuracy (Gen. Top- 1/oo//TTo-55. Our SCA method consistently achieves the best balance across all metrics.
Synthesis Model
Image Encoder
DreamSim
Vendit
CLIP-T1
Gen. Top-1
Gen. Top-5
ELITE 84
CLIP-B
0.5239
29.37
0.2816
20.11%
30.57%
B4T 26
CLIP-L
0.5384
31.22
0.2819
2.26%
5.82%
IP-Adapter
CLIP-H
0.5037
31.92
0.3007
26.18%
48.28%
FastComposer
86
CLIP-L
0,5194
30.82
0,2944
4.06%
9,94%
SDXIL-Finetuned
CLII--
0.4812
31.85
0.2721
36.53%
62.00%
SDXI.-Finetuned
DINOv2-B
0.4537
32.24
0.2803
58.45%
80.84%
SDXL-Finetuned (Ours) SCA
0.4293
34.48
0.3106
67.28 %
87.48 %
Table 12: Comparison with off-theeshheff diffusion adapters iNaturalist. Our SCA method achieves substantial improvements over pre-trained adapters and alternative visual encoders across all metries.
Following the evaluation protocol in Table 5 the main paper, we conduct experiments on the iNaturalist dataset. For each method, generate synthetic data using the respective adapter and measure: (1) concept preservation using DreamSim 23). (2) sample diversity using the Vendi
Score 22 (3) text-image alignment using CLIP-T (61), and (4) generation quality using Top-I and Top-5 accuracy from an external classifier (a strong model from the TIMM leaderboard), We additionally include ablations using SDXL fine-tuned with CLIP and DINOv2 encoders (without SCA training procedure) to isolate the contribution of our proposed loss function.
Table 12
presents comprehensive comparison. Our method consistently outperforms all off-the-shelf adapters across every metric. Notably, we achieve a 41.1% improvement Gen. Top-1 accuracy over the best off-the-sheel adapter (IP-Adapter: 26.18% vs. Ours: 67.28%), demonstrating the critical importance of domain-specific adaptation for ine-grained visual tasks.
Pre-trained adapters such as IP-Adapter and ELITE, while effective for general- purpose image generation, show poor generation quality on fine-grained classification tasks. Even IP-Adapter with CLIP-H achieves only 26.18% Top-1 accuracy, indicating that the generated images frequently do not preserve the correct fine- mne-grained category. This suggests that generic adapters lack the specialized visual understanding required for subtle inter-class distinctions in domains like species

---

Model
IN-LT
iNat
Cars
CUB
Flowers
Aircraft
Food
Dogs
Average
SCA Probing
69.64
65.74
94.31
90.24
99.17
85.22
93.97
90.93
86,
SCA Augmented (Ours)
79.17
74.11
95.17
91.47
99.92
88.42
95.61
91.84
89.46
Improvement
+9.53
+8.37
+0.86
+1.23
+0.75
+3.20
+1.64
+0.91
+3.31
Table 13: Comparison of direct SCA feature classification vs. SCA-guided generative augmentation. Classifica- tion accuracy (%) on eight datasets demonstrates that generative augmentation provides substantial improvements over direct feature use.
identification. Fine-tuning SDXL with CLIP-B embeddings (without our SCA training) yields 36.53% Top-1 accuracy, already surpassing all off-the-shelf methods. This demonstrates that task-specific adaptation of the synthesis model is essential for fine-gra tnn-grained domains. Switching to DINOv2-B embeddings further improves performance to 58.45%, highlighting the importance of encoder choice. Our proposed method, which combines DINOv2- b-based initialization with angular margin loss training, achieves 67.28% Top-1 accuracy, which is an 8.83% improvement over DINOv2 alone. This improvement, coupled with the best CLIP-T score (0.3106) and highest diversity (34.48 Vendi Score), demonstrates that our SCA training procedure effectively learns to encode salient visual concepts while maintaining text alignment and sample diversity.
Direct Classification with SCA Features G
Our approach can be understood through the lens of knowledge distillation: the SCA embedding model encodes task-specific visual priors, which are then transferred downstream classifiers via the synthesis model. The synthesis model, leveraging its world knowledge, generates images with diverse incidental features (backgrounds, lighting, poses) while preserving salient concepts. This process enables the downstream classifier to learn robust representations that focus on class-discriminative features rather than spurious correlations.
A natural question arises: could we bypass the synthesis stage entirely and use SCA embeddings directly for classification? Despite conceptually appealing, this approach would forfeit the primary benefit of our method that leverages the foundation synthesis model's world knowledge to generate diverse training samples that improve classifier robustness.
To investigate this question empirically, we compare two approaches: (1) SCA Probing: training linear classification head frozen SCA features, and (2) SCA Augmented: our full method using SCA-guided generative augmentation. For fair comparison, we use OpenAl CLIP-B/16 as the backbone for SCA training, matching the initialization used for downstream classifiers in our main experiments. As shown in Table [3] our full generative augmentation method outperforms direct SCA feature classification by an average of 3.31% across eight datasets. The improvement is particularly pronounced on challenging datasets: ImageNet-LT (+9.53%) and iNaturalist (+8.37%), both of which feature long-tailed distributions with limited samples for many classes.
important to emphasize that SCA training is not a silver bullet for isolating salient concepts with perfect precision. Rather, it is effective mitigation strategy that reduces conflicts between image and text prompts during synthesis, helping achieve a better fidelity-diversity trade-off in generated data. The synthesis model remains the primary source of distributional diversity that drives downstream classifier robustness.
5


Reviews:
[{"review_id": "Uxu2hIwzvR", "reviewer": "Reviewer_wJKD", "summary": "This paper presents a framework for generative data augmentation (GDA) that effectively addresses the fidelity-diversity trade-off by introducing a salient concept-aware (SCA) image embedding model. The proposed method disentangles essential visual features from irrelevant attributes, enabling better alignment with text prompts while preserving class-discriminative details. The authors demonstrate significant improvements in classification accuracy across multiple fine-grained datasets, particularly in long-tail settings.", "questions": "According to the W.1-2.", "limitations": "N/A", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Proposes a novel salient concept-aware (SCA) embedding model that effectively disentangles essential class-discriminative features from irrelevant image attributes (e.g., background), directly addressing the fidelity-diversity trade-off in generative data augmentation.\n2. Eliminates reliance on manual interventions (e.g., segmentation masks) or subject-specific optimization, enhancing scalability for real-world applications.\n3. Demonstrates state-of-the-art results across eight fine-grained datasets under diverse settings (conventional, long-tail, few-shot), with significant accuracy gains (up to +6.5%).\n\nWeaknesses:\nThis paper does not seem to have any major flaws.\n1. Focuses exclusively on fine-grained visual tasks; applicability to broader or non-visual domains remains unverified.\n2. Fails to analyze computational costs of training the SCA embedding model, a critical factor for adoption.", "was_revised": false, "final_justification": "", "rebuttal": "**Question**: *Focuses exclusively on fine-grained visual tasks; applicability to broader or non-visual domains remains unverified.*\n\n**Answer**: Our focus on fine-grained classification tasks is motivated by practical challenges in niche domains where expert models must be trained on domain-specific datasets that are often small or class-imbalanced. GDA is particularly valuable in these settings, as it can generate faithful and diverse samples by leveraging world priors learned from synthesis models, thereby improving robustness of downstream classifier. However, our current synthesis pipeline is built on a diffusion model (SDXL), which is not well-suited for complex vision tasks such as text-rich VQA or GraphQA. There are other interesting experiments for our method to demonstrate: a) broader applicability to diverse domains, b) application to multi-modal models.\n\n**a) broader applicability to diverse domains**: \nWe perform a few-shot experiment on Places365, a dataset for abstract scene understanding rather than the object-centric datasets we primarily focused on. The evaluation is done on its validation set. We follow the same the N-way 10-shot experiment setting in Table 2 of the manuscript. The result is shown below:\n\nMethod    | Vanilla | CutMix  | Mixup  | Da-Fusion | Diff-Mix  | Ours\n----------|---------|---------|--------|-----------|-----------|-------\nTop-1     | 35.32   | 36.23   | 35.53  | 36.57     | 37.53     | **39.06**\nTop-5     | 64.15   | 65.62   | 64.47  | 65.88     | 66.91     | **68.69**\n\n*Table 1: Classification accuracy (%) using 10-shot training samples on Places 365 validation set. Our method out-performs peer method by 1.53% and 1.78% in Top-1 and 5 accuracy, respectively, demonstrating that our method is also effective on datasets of abstract scene as well.*\n\n**b) application to multi-modal models**:\nClassification is still an important task for modern multi-modal models that often require fine-tuning domain-specific dataset to achieve production-level performance. To demonstrate the effectiveness of our generation method in this context, we explore multi-modal visual classification by performing supervised fine-tuning on an MLLM using FGVC datasets. In this setup, we tune only the projector and vision encoder and keep the LLM frozen. The classification problem is reformulated as a 26-choice QA task, where the ground-truth label is always included among the choices. An prompt example is provided below:\n*\"You are an expert classifier. Based on the visual evidence in the image, select the most appropriate category. Please choose ONE of the following:\nA. beef_tartare\nB. creme_brulee\n...\nZ. tacos\nPlease respond ONLY with your answer wrapped in this format: \\<answer\\> LETTER \\<\\/answer\\>.\"*\n\nMethod                | CUB     | Aircrafts | Flowers  | Cars    | Dogs    | Food   | Average\n----------------------|---------|-----------|----------|---------|---------|--------|--------\nZero-shot             | 79.44%  | 58.78%    | 82.91%   | 87.58%  | 78.48%  | 90.31% | 79.58%\nFine-tuned - Diff-Mix | 89.37%  | 66.92%    | 94.71%   | 95.30%  | 87.14%  | 83.23% | 86.11%\nFine-tuned - Ours     | **90.49**%  | **79.31**%    | **95.30**%   | **96.57**%  | **87.33**%  | **91.53**% | **90.09**%\n\n*Table 2: Classification accuracy (%) under N-way 10-shot setting with Qwen2.5-VL-7B backbone. We report the zero-shot and fine-tuned performance. The zero-shot performance is not satisfactory on majority of the FGVC datasets. When fine-tuning the model with augmented data, our method out-performs Diff-mix by 3.98% on average across 6 FGVC datasets.*\n\n\n**Question**: *Fails to analyze computational costs of training the SCA embedding model, a critical factor for adoption.*\n\n**Answer**: Our method involves two training stages: (1) training the SCA embedding model, and (2) fine-tuning the synthesis model using SCA embeddings; training the SCA model requires about 20% of the time needed to fine-tune the synthesis model. As shown in the table below, on an 8×A100 GPU setup, training both components takes approximately 20–30 minutes for smaller-scale FGVC datasets of size ~10K, and up to 3 days for iNat2021 with 2.7M images, the largest dataset we have experimented with.  We argue that this cost is practical for most application use-cases, especially for scenarios where curating high-quality in-domain data is prohibitively expensive. Our method offers a promising solution, giving consistent performance improvements over baseline GDA methods.\n\nDataset | FGVC       | iNat2018 | ImageNet1K | iNat2021\n--------|------------|----------|------------|------------\nSize    | ~10K       | 440K     | 1.3M       | 2.7M\nTime    | 20-30 mins | 19 hours | 25 hours   | ~3 days"}, {"review_id": "XlYT535zjz", "reviewer": "Reviewer_fpDS", "summary": "This paper proposes an image generation framework for data augmentation on image and text prompts. The main component of the design is a \"salient concept-aware\" image embedding model. The idea is to preserve key attributes of the input image while reducing the influence of the background features. This is achieved by training the image embeddings with additional penalty terms. Experiments are conducted on eight fine-grained image datasets, and the proposed method is compared with other data augmentation methods.", "questions": "- Figure 2: for image (c), it seems the proposed model SCA learns some sort of style transfer--for the given image (f), it treats the black mask as the texture of the fly and draws a black fly/moth in image (c). This is slightly different from what is expected if the learned embeddings are truly about either the fly or the background flowers. Can the authors clarify what's happening here when a salient object is masked out? \n\n- Please see the discussion on Weaknesses and clarify how the proposed loss term enables the learning of 'salient' features. \n\n- In some experiments (such as Section 4.3), only a small subset of the dataset is selected. Is there a reason not to test on all fine-grained datasets? \n\nOverall, I like the simplicity of the approach and, in contrast, how much improvement it can achieve in the long-tail classification tasks. It shows promising results that might inspire more future research in addressing data augmentation for long-tail classes.", "limitations": "No discussion on limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n- Generative data augmentation is an important task. Trying to address it to generate fine-grained categories is a good motivation. \n- The experiment section is fairly comprehensive, covering a wide range of methods and benchmarks, as well as essential ablations. \n- The paper focuses on obtaining better image embeddings, without the need to make substantial changes to the generation model itself. This makes the approach flexible, and it is possible to adapt to other generation models. \n\n### Weaknesses\n- The core of this paper is the loss term in the Salient Concept-Aware Embedding Model (Eq. 1). It looks like a tuned version of a contrastive loss (based on cosine similarity with margins) in a contrastive learning framework. It is unclear to me where the 'salient' part comes from in this formulation: I think it is closer to a max-margin positive feature vs negative feature learning scheme. \n- The 0.73% accuracy improvement in the conventional setting is underwhelming. One would expect that having a good data augmentation to obtain more diverse data should improve the results more, even in the traditional setting without the focus on the few-shot or long-tail cases.", "was_revised": true, "final_justification": "After reading the rebuttal, I think the paper is in a better shape with the addition of the experiments and clarifications. I recommend the acceptance of the paper.", "rebuttal": "**Question**: *The core of this paper is the loss term in the Salient Concept-Aware Embedding Model (Eq. 1). It looks like a tuned version of a contrastive loss (based on cosine similarity with margins) in a contrastive learning framework. It is unclear to me where the 'salient' part comes from in this formulation: I think it is closer to a max-margin positive feature vs negative feature learning scheme.*\n\n**Answer**: Salient concept learning is from the combination of the proposed loss and the structure of the training data. For example, consider two images of the same bird species, one standing on a tree, the other on a rock. Cross-entropy loss is happy as long as the prediction is correct, even if the model encodes incidental features (e.g., trees or rocks). In contrast, our margin-based loss encourages intra-class compactness in the embedding space, penalizing incidental variations that introduce noise.\n\nWe would like to note that SCA is not a silver bullet for isolating salient concepts. It is an effective mitigation strategy to reduce conflicts between image and text prompts during synthesis, eventually helping achieve a better fidelity-diversity trade-off in the generated data.\n\nWe agree with the reviewer that our formulation can be viewed as a form of max-margin learning. But we want to highlight the role of the angular loss: image embeddings are first normalized before being fed into the adapter and cross-attention layers of the diffusion model. This normalization stabilizes gradient flow during fine-tuning and better aligns the image embeddings with the text embeddings. Our angular margin–based loss is specifically designed to optimize these normalized embeddings (see Lines 144–145 of the manuscript). In contrast, embeddings trained with cross-entropy typically require post-hoc normalization and gives inferior performance in our experiments (see Table 6 of the manuscript).\n\n**Question**: *The 0.73% accuracy improvement in the conventional setting is underwhelming. One would expect that having a good data augmentation to obtain more diverse data should improve the results more, even in the traditional setting without the focus on the few-shot or long-tail cases.*\n\n**Answer**: We agree with the reviewer that a 0.73% improvement over peer methods in the conventional setting is modest in absolute terms. Another perspective is to compare relative gains: the current SOTA GDA method improves performance over the vanilla baseline by 0.855%, and our method achieves a 1.585% improvement. This is an 85% relative performance gain over the SOTA method, which is not marginal.\n\nIt is also true that the conventional setting is more challenging to improve: with fixed task complexity, as datasets become more balanced and diversified, there is less room for robustness improvement through augmentation (e.g., a model pretrained on web-scale data may show little to no further gain on the easy task distinguishing cats and dogs). Our work is motivated by practical challenges in niche domains, where expert models must be trained on domain-specific datasets that are often small, class-imbalanced, or updated incrementally (e.g., with new categories over time). In these settings, our method is a good choice for practitioners as it consistently outperforms baseline GDA methods across conventional, few-shot, long-tail, and out-of-distribution classification tasks.\n\n**Question**: *Figure 2: for image (c), it seems the proposed model SCA learns some sort of style transfer—for the given image (f), it treats the black mask as the texture of the fly and draws a black fly/moth in image (c). This is slightly different from what is expected if the learned embeddings are truly about either the fly or the background flowers. Can the authors clarify what's happening here when a salient object is masked out?*\n\n**Answer**:\nThis is a very interesting observation that we didn’t pay attention to before. We suspect that the SCA model is influenced by learned priors from the training data, driving it to interpret the black masked region at the center into one of the species from the iNaturalist dataset. To validate this we computed the SCA embedding of the masked image (Fig. 2(f)) and compared it with the average class embeddings across all species in iNat dataset. The top nearest matches were:\n\n1. Atolmis rubricollis (Insects), cosine similarity: 0.31\n2. Lucidota atra (Insects), cosine similarity: 0.19\n\nBoth of them are black-colored insect species. Given this result, we agree with the reviewer that our model’s behavior is analogous to style transfer. Furthermore, we think the SCA embedding encodes key foreground information from the masked image (Fig. 2(f)) including style, color, textual of the concept of a dark insect. Conditioned on those, the synthesis model is able to reconstruct the final generated image (Fig. 2(c)).\n\n\n**Question**: *In some experiments (such as Section 4.3), only a small subset of the dataset is selected. Is there a reason not to test on all fine-grained datasets?*\n\n**Answer**: We used the two largest, representative and most challenging datasets (ImageNet and iNaturalist) in our experiment setting to analyze the quality of the generated images in Table 5 of Section 4.3.\nWe complete the Table 5 with the remaining 6 FGVC datasets. We will include the following results in the supplementary of the revised manuscript.\n\n\nDataset   | Model         | DreamSim↓ | Vendi↑  | CLIPT↑  | Gen. Top1 | Gen. Top5\n----------|---------------|-----------|---------|---------|-----------|------------\nFlowers   | Real-Guidance | 0.3307    | 15.16   | **0.3236**  | 87.31%    | 95.30%\nFlowers  | DA-Fusion     | 0.3260    | 13.06   | 0.2795  | 88.24%    | 96.22%\nFlowers  | Diff-Mix      | 0.3195    | 14.20   | N/A     | 91.83     | 99.00%\nFlowers  | SCA           | **0.3051**    | **15.31**   | 0.3089  | **95.27**%    | **99.88**%\n&nbsp;    |               |           |         |         |           | \nCars      | Real-Guidance | 0.4382    | **8.01**    | **0.3627**  | 54.27%    | 82.93%\nCars      | DA-Fusion     | 0.4263    | 7.59    | 0.3169  | 60.85%    | 85.60%\nCars      | Diff-Mix      | 0.4035    | 7.72    | N/A     | 85.46%    | 98.69%\nCars      | SCA           | **0.3880**    | 7.96    | 0.3411  | **86.60**%    | **99.07**%\n&nbsp;    |               |           |         |         |           | \nAircrafts | Real-Guidance | 0.4676    | 7.59    | **0.3411**  | 25.67%    | 65.99%\nAircrafts | DA-Fusion     | 0.4549    | 6.54    | 0.2974  | 33.58%    | 71.35%\nAircrafts | Diff-Mix      | 0.4230    | 7.04    | N/A     | 49.46%    | 81.96%\nAircrafts | SCA           | **0.4001**    | **7.61**    | 0.3167  | **60.17**%    | **92.95**%\n&nbsp;    |               |           |         |         |           | \nCUB       | Real-Guidance | 0.4584    | **11.91**   | **0.3479**  | 63.03%    | 93.75%\nCUB       | DA-Fusion     | 0.4624    | 11.81   | 0.3033  | 62.14%    | 90.97%\nCUB       | Diff-Mix.     | 0.4379    | 11.49   | N/A     | 69.53%    | 96.49%\nCUB       | SCA           | **0.4092**    | 11.89   | 0.3283  | **73.66**%    | **97.64**%\n&nbsp;    |               |           |         |         |           | \nFood      | Real-Guidance | 0.4833    | **21.44**   | **0.3413**  | 87.71%    | 99.03%\nFood      | DA-Fusion     | 0.4796    | 19.46   | 0.2974  | 87.56%    | 98.41%\nFood      | Diff-Mix.     | 0.4513    | 20.13   | N/A     | 90.16%    | 99.22%\nFood      | SCA           | **0.4374**    | 21.31   | 0.3289  | **92.48**%    | **99.84**%\n&nbsp;    |               |           |         |         |           | \nDogs      | Real-Guidance | 0.4731    | 37.92   | **0.3668**  | 79.38%    | 96.09%\nDogs      | DA-Fusion     | 0.4569    | 32.07   | 0.3011  | 79.74%    | 97.27%\nDogs      | Diff-Mix      | 0.4274    | 34.73   | N/A     | 81.24%    | 97.40%\nDogs      | SCA           | **0.4105**    | **37.99**   | 0.3439  | **83.78**%    | **98.26**%"}, {"review_id": "RjbFv2Lcyf", "reviewer": "Reviewer_MqZv", "summary": "The paper presents an improved method for generative data augmentation that focuses on generating diverse images of the same class while preserving only the relevant class features, while discarding unimportant information, such as the background or position.\nThe method is based on fine-tuning a diffusion model that is conditioned on \"salient concept-aware\" image embeddings, allowing the model to generate diverse images of the same input class.\nThe method is validated through different evaluations, including conventional and few-shot classification.", "questions": "Some things are not completely clear:\n- Is the Domain Expert Embedding Model trained together with the diffusion model? \n- Is it trained separately for each task?\n- L. 128-131: Why CLIP Image features are not suitable for the task? Unlike DINO, CLIP is trained under contrastive settings between images and **text** such that it captures the important concepts in the image.\n- I wonder how the method improves over augmentation that utilizes other pre-trained diffusion adapters, e.g.:\n[1] IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models, Ye et al.\n[2] BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing, Li et al.\n[3] ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation, Wei et al.", "limitations": "Mentioned limitations were not discussed.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The method is simple and effective in classification tasks. \nHowever, the method is expensive to train; IIUIC, it requires fine-tuning of the SCA and the diffusion model for each classification task.\nComparison to other off-the-shelf adapters is required.\nAdditionally, I found the scope of the method to be limited: could the proposed data augmentation help to improve generative personalization tasks (e.g., 1, 2), or other discriminative tasks?\n\n[1] An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion, Gal et al.\n[2] Multi-Concept Customization of Text-to-Image Diffusion, Kumari et al.", "was_revised": true, "final_justification": "My concerns have been addressed in the rebuttal and I raise my score to 4.", "rebuttal": "**Question**: *However, the method is expensive to train; IIUIC, it requires fine-tuning of the SCA and the diffusion model for each classification task.*\n\n**Answer**: We acknowledge that our approach is more computationally intensive than classical augmentation or existing generative data augmentation (GDA) methods, as it involves training a SCA embedding model and fine-tuning a diffusion model. As shown in the table below, on an 8×A100 GPU setup, training takes approximately 20–30 minutes for smaller fine-grained datasets (e.g., ~10K images), and up to 3 days for large-scale datasets such as iNaturalist 2021 (2.7M images).\nWe argue that this cost is practical for most application use-cases, especially for scenarios where curating high-quality in-domain data is prohibitively expensive. Our method offers a promising solution, giving consistent performance improvements over baseline GDA methods.\n\nDataset | FGVC       | iNat2018 | ImageNet1K | iNat2021\n--------|------------|----------|------------|------------\nSize    | ~10K       | 440K     | 1.3M       | 2.7M\nTime    | 20-30 mins | 19 hours | 25 hours   | ~3 days\n\n**Question**: *Comparison to other off-the-shelf adapters is required.*\n\n**Question**: *I wonder how the method improves over augmentation that utilizes other pre-trained diffusion adapters, e.g.: [1] IP-Adapter [2] BLIP-Diffusion [3] ELITE*\n\n**Answer**: We compare our method with IP-Adapter [1], ELITE [3], E4T [4], , FastComposer [5].\n\nDataset | Synthesis Model  | Embedder  | DreamSim↓ | Vendi↑  | CLIPT↑  | Gen. Top1 | Gen. Top5\n--------|------------------|-----------|-----------|---------|---------|-----------|------------\niNat    | ELITE [3]           | Clip-B    | 0.5239    | 29.37   | 0.2816  | 20.11%    | 30.57%\niNat    | E4T [4]             | Clip-L    | 0.5384    | 31.22   | 0.2819  | 2.26%     | 5.82%\niNat    | IP-Adapter [1]       | Clip-H    | 0.5037    | 31.92   | 0.3007  | 26.18%    | 48.28%\niNat    | FastComposer [5]     | Clip-L    | 0.5194    | 30.82   | 0.2944  | 4.06%     | 9.94%\niNat    | SDXL-Finetuned   | Clip-B    | 0.4812    | 31.85   | 0.2721  | 36.53%    | 62.00%\niNat    | SDXL-Finetuned   | DINOv2-B  | 0.4537    | 32.24   | 0.2803  | 58.45%    | 80.84%\niNat    | SDXL-Finetuned   | SCA       | **0.4293**    | **34.48**   | **0.3106**  | **67.28**%    | **87.48**%\n\nIn line with Table 5 of the manuscript, we conduct evaluation using the iNaturalist dataset. For each method, we generated synthetic data and measured concept preservation using DreamSim, diversity with Vendi, and text–image alignment using CLIP-T. Additionally, we evaluate Top-1 and Top-5 accuracy on the synthetic data using an external classifier from the TIMM leaderboard (denoted as Gen. Top-1 and Gen. Top-5). Our method consistently outperforms other off-the-shelf adapters across all metrics, with 41.1% and 2.56 improvements on Gen. Top-1 accuracy and CLIPT scores, respectively.\n\n*[1] Ye et al., “IP-Adapter: Text-Compatible Image Prompt Adapter for Text-to-Image Diffusion Models,” arXiv:2308.06721, 2023.*\n\n*[3] Wei et al., “ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation,” ICCV 2023.*\n\n*[4] Gal et al., “Encoder-Based Domain Tuning for Fast Personalization of Text-to-Image Models,” TOG 2023.*\n\n*[5] Xiao et al., “FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention,” arXiv:2305.10431, 2023.*\n\n**Question**: *Is the Domain Expert Embedding Model trained together with the diffusion model?*\n\n**Answer**: The domain expert embedding model is trained separately and kept frozen during the diffusion model fine-tuning.\n\n**Question**: *Is it trained separately for each task?*\n\n**Answer**: Yes, the SCA embedding model as well as the diffusion model are trained separately for each task.\n\n**Question**: *L. 128-131: Why CLIP Image features are not suitable for the task? Unlike DINO, CLIP is trained under contrastive settings between images and text such that it captures the important concepts in the image.*\n\n**Answer**: Our experimental findings show that DINOv2 consistently outperforms CLIP by a significant margin across all eight datasets. We include the iNaturalist results alongside the off-the-shelf adapter results in the table above. Here we fine-tune the synthesis model to be compatible with each image embedding model, following the same procedure used with SCA. By switching from DINOv2 to Clip, the CLIPT score is dropped by 0.0082 and the Gen. Top-1 score is dropped by 21.92%. We suspect the reason is that CLIP is optimized for global semantic consistency with high-level language descriptions, making it less effective for fine-grained visual tasks such as image generation that require understanding of low-level details. In contrast, DINOv2 is trained with a self-supervised objective targeted for vision task only. This observation is also supported by recent work [6], where they show in Table 2 that aligning diffusion model representations with DINO embeddings leads to superior performance in generative tasks, compared to CLIP embeddings.\n\n*[6] Zhang et al., “Representation Alignment for Generation: Training Diffusion Transformers is Easier Than You Think,” ICLR 2025.*\n\n**Question**: *Additionally, I found the scope of the method to be limited: could the proposed data augmentation help to improve generative personalization tasks (e.g., 1, 2), or other discriminative tasks?*\n\n**Answer**:\n[Generative Personalization Tasks] Our method could potentially improve personalized image generation methods that rely on image encoders to extract image prompt information (e.g., [1], [3–5]). Prior work has mainly focused on redesigning multimodal attention in diffusion models, and our approach is orthogonal to them as we improve the image encoder to reduce conflicts between irrelevant visual features and text prompts, resulting in images that more faithfully align with the intended concepts.\n\nDue to the new rebuttal policy, we cannot include qualitative comparisons. Informally, we observe that our method achieves better text‑prompt alignment than IP‑Adapter and greater visual quality than ELITE. For example, given the prompt \"Brilliant purple flower, aged brick wall, gentle shadows\" and an input image of a violet on green grass, ELITE outputs a pink-purple blend which does not look like flowers. IP-Adapter outputs a counterfactual image of a violet on a stone road with surrounding green grass. Our method faithfully generates a violet near a red brick wall under sunlight.\n\nThe primary focus of this paper is on discriminative tasks, motivated by practical challenges in niche domains where expert models are only trained on small or class‑imbalanced domain-specific datasets, a regime where our method shines. Extending our approach to generative personalization is an exciting direction for future work.\n\n[Discriminative Vision Tasks] Our current synthesis pipeline is built on a diffusion model (SDXL), which is not well-suited for complex vision tasks such as text-rich VQA or GraphQA. There are other interesting experiments for our method to demonstrate: a) broader applicability to diverse domains, b) application to multi-modal models.\n\n**a) broader applicability to diverse domains**:\nWe perform a few-shot experiment on Places365, a dataset for abstract scene understanding rather than the object-centric datasets we primarily focused on. The evaluation is done on its validation set. We follow the same the N-way 10-shot experiment setting in Table 2 of the manuscript. The result is shown below:\n\nMethod    | Vanilla | CutMix  | Mixup  | Da-Fusion | Diff-Mix  | Ours\n----------|---------|---------|--------|-----------|-----------|-------\nTop-1     | 35.32   | 36.23   | 35.53  | 36.57     | 37.53     | **39.06**\nTop-5     | 64.15   | 65.62   | 64.47  | 65.88     | 66.91     | **68.69**\n\nWe report classification accuracy (%) using 10-shot training samples on Places 365 validation set. Our method out-performs peer method by 1.53% and 1.78% in Top-1 and 5 accuracy, respectively, demonstrating that our method is also effective on datasets of abstract scene as well.\n\n**b) application to multi-modal models**:\nClassification is still an important task for modern multi-modal models that often require fine-tuning domain-specific dataset to achieve production-level performance. To demonstrate the effectiveness of our generation method in this context, we explore multi-modal visual classification by performing supervised fine-tuning on an MLLM using FGVC datasets. In this setup, we tune only the projector and vision encoder and keep the LLM frozen. The classification problem is reformulated as a 26-choice QA task, where the ground-truth label is always included among the choices. An prompt example is provided below:\n*You are an expert classifier. Based on the visual evidence in the image, select the most appropriate category. Please choose ONE of the following:\nA. beef_tartare\nB. creme_brulee\n...\nZ. tacos\nPlease respond ONLY with your answer wrapped in this format: \\<answer\\> LETTER \\<\\/answer\\>.*\n\nMethod                | CUB     | Aircrafts | Flowers  | Cars    | Dogs    | Food   | Average\n----------------------|---------|-----------|----------|---------|---------|--------|--------\nZero-shot             | 79.44%  | 58.78%    | 82.91%   | 87.58%  | 78.48%  | 90.31% | 79.58%\nFine-tuned - Diff-Mix | 89.37%  | 66.92%    | 94.71%   | 95.30%  | 87.14%  | 83.23% | 86.11%\nFine-tuned - Ours     | **90.49**%  | **79.31**%    | **95.30**%   | **96.57**%  | **87.33**%  | **91.53**% | **90.09**%\n\nWe report classification accuracy (%) under zero-shot and N-way 10-shot setting with Qwen2.5-VL-7B backbone. The zero-shot performance is not satisfactory on majority of the FGVC datasets. When fine-tuning the model with augmented data, our method out-performs Diff-mix by 3.98% on average across 6 FGVC datasets."}, {"review_id": "6tp3A0lS0Y", "reviewer": "Reviewer_Vnct", "summary": "This paper addresses the challenge of balancing fidelity and diversity in generative data augmentation (GDA) methods conditioned on both image and text prompts. The key idea is to use a salient concept-aware (SCA) image embedding model that emphasizes class-relevant features while suppressing irrelevant background or context information. The method first independently trains domain-specific salient concept-aware embedding model with a penalty to improve both the intra-class compactness and inter-class discrepancy. Then the method trains the synthesis model, taking frozen image and text embeddings as inputs and only tunes the adapters. This approach does not require intensive subject-specific optimization or additional input processing such as saliency segmentation. Experiments on eight fine-grained visual categorization datasets show improved classification accuracy compared with prior GDA methods, particularly in challenging long-tail and out-of-distribution scenarios. Extensive ablation studies further demonstrate the advantages and stability of the proposed method.", "questions": "- In the introduction, the paper refers to synthetic image generation as a form of distillation. Based on this, can generative data augmentation be understood as a process where knowledge from the salient concept-aware embedding model is distilled into the classifier via the synthesis model? From this perspective, I am curious whether using the features from the salient concept-aware embedding model directly for classification would yield results comparable to those obtained through generative data augmentation.\n\n- In the comparisons for conventional, long-tail, and OOD classification, is the amount of synthetic data generated by each method kept consistent? If not, I would appreciate it if the authors could report the amount of synthetic data produced by each method.", "limitations": "No. The authors should include technical limitations and potential social impacts.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is clearly written and provides valuable high-level insights. The figures in the main text significantly aid understanding, and the supplementary material offers comprehensive implementation details.\n\n- The work identifies a practical and impactful limitation of existing GDA methods—namely, that their embeddings often entangle information that is less relevant to the concept of interest.\n\n- The proposed two-stage training strategy, which first finetunes a domain-specific salient concept-aware embedding model and then trains the synthesis model, is effective. The use of a novel penalty to improve both intra-class compactness and inter-class separation during embedding model finetuning is particularly interesting. The design of the conditional synthesis model and image prompt adapter is also well-motivated, and the paper provides clear strategies for constructing and filtering training data.\n\n- The experiments are thorough, with extensive comparisons across multiple datasets demonstrating state-of-the-art performance. In addition, detailed ablation studies convincingly validate the effectiveness of each component.\n\n**Weaknesses**\n\n- The proposed approach still requires finetuning both the salient concept-aware embedding model and the synthesis model for each new dataset, which limits its generalizability and falls short of achieving truly optimization-free general data augmentation.", "was_revised": true, "final_justification": "I believe all of my concerns have been fully addressed. Therefore, I am willing to raise my score.", "rebuttal": "**Question**: *In the introduction, the paper refers to synthetic image generation as a form of distillation. Based on this, can generative data augmentation be understood as a process where knowledge from the salient concept-aware embedding model is distilled into the classifier via the synthesis model? From this perspective, I am curious whether using the features from the salient concept-aware embedding model directly for classification would yield results comparable to those obtained through generative data augmentation.*\n\n**Answer**: Our work is motivated by practical challenges that researchers often need to train expert classifiers on domain-specific datasets that are typically limited in quantity, class-imbalanced, or require adaptation to new classes with very few examples. To address this problem, we use foundation synthesis model with world knowledge to generate images with diverse incidental features to improve the robustness of the downstream classifier that can focus on the salient concept. We can think of it as a form of indirect knowledge distillation.\n\nHowever, existing text-to-image or GDA approaches often fail to generate faithful images due to either the limitations of textual descriptions or inability to incorporate task priors. The role of SCA is to introduce the task priors to the synthesis model, achieving optimal fidelity-diversity trade-off. In this way, the foundation synthesis model serves as a task-aware source of world knowledge that the downstream classifier can effectively learn from.\n\nIt’s also important to note that the proposed SCA training mechanism is not a silver bullet for learning to isolate salient concepts with exceptional precision. It is an effective mitigation strategy to reduce conflicts between image and text prompts during synthesis, eventually helping achieve a better fidelity-diversity trade-off in the generated data.\n\nTherefore, we suspect that the performance of features from SCA models alone cannot be comparable to that of our augmentation method.\nTo validate this, we evaluate the performance of the SCA embedding model by probing its frozen features with classification head. For a fair comparison, we used the OpenAI CLIP‑B/16 backbone for SCA probing (borrowed from the experiment requested by Reviewer MqZv to compare DINO with CLIP), since our downstream classifiers are also trained from OpenAI CLIP‑B/16.\n\nModel               | IN-LT  | iNat   | CAR196 | CUB200 | Flower102 | Aircraft100 | Food101 | Dogs120 | Average\n--------------------|--------|--------|--------|--------|-----------|-------------|----------|--------|----------\nSCA Probing         | 69.64% | 65.74% | 94.31% | 90.24% | 99.17%    | 85.22%      | 93.97%   | 90.93% | 86.15%\nSCA Augmented       | **79.17**% | **74.11**% | **95.17**% | **91.47**% | **99.92**%    | **88.42**%      | **95.61**%   | **91.84**% | **89.46**%\n\nOur augmentation method out-performs the SCA embedding by an average of 3.31% over 8 datasets.\n\n**Question**: *In the comparisons for conventional, long-tail, and OOD classification, is the amount of synthetic data generated by each method kept consistent? If not, I would appreciate it if the authors could report the amount of synthetic data produced by each method.*\n\n**Answer**: For CutMix and Mixup, we apply it to every training sample in the mini-batch.\nFor GDA methods, we generated synthetic data with their public code and use them to train our downstream classifiers. The classifier training protocol outlined in section 4.1 is the same for all methods, except for the synthetic-to-real ratio K and the probability of sampling synthetic images P during training. We follow the settings from original papers for the choices of K and P described below:\n\n* For our method, we use P=20% for ImageNet and iNaturalist, and P=50% for all other FGVC dataset. In figure 4 of the manuscript, we show that our method is robust for P in the range from 20% to 50%. We set K=5 all datasets.\n* For Real-Guidance, we use P=50% and K=5 for all datasets.\n* For Da-Fusion, we use P=50% and K=10 for all datasets.\n* For Diff-Mix, we use P=10% and K=5 for all datasets.\n\nIn addition, we guarantee that at least 250 synthetic data are generated for each category, applicable to all methods. This is to make sure that enough data are generated for few-shot experiments and under-represented tail categories in LT experiments.\n\n\n**Question**: *The proposed approach still requires finetuning both the salient concept-aware embedding model and the synthesis model for each new dataset, which limits its generalizability and falls short of achieving truly optimization-free general data augmentation.*\n\n**Answer**: We acknowledge that the proposed approach requires fine-tuning both the salient concept-aware embedding model and the synthesis model for each new dataset.\nHowever, it is important for our method to remain functional for samples from previously unseen or underrepresented classes within the same domain as the training data. For example, if the training dataset contains various dog breeds but excludes bulldogs, the model should still recognize a bulldog as a distinct and relevant class and accurately preserve its concept during the synthesis process. This is partially demonstrated in our LT and OOD experiments in Table 4 of the manuscript, and we will dive deeper in this rebuttal.\n\nWe consider an “open-set” setup where our embedding and synthesis models were trained using only 80% of randomly selected species from the iNat training set, and the remaining 20% of species were held out and only used for evaluation. We compare our method with two peer personalized image generation methods as recommended by Reviewer MqZv.\n\nDataset        | DreamSim↓ | Vendi↑ | CLIPT↑  | Gen.Top1  | Gen.Top5\n---------------|-----------|--------|---------|-----------|----------\nELITE [2]      | 0.5226    | 29.15  | 0.2837  | 14.10%    | 25.47%\nIP-Adapter [1] | 0.4993    | 31.80  | 0.2993  | 23.76%    | 48.34%\nOurs           | **0.4419**    | **34.28**  | **0.3033**  | **60.35**%    | **84.19**%\n\nThe Top-5 accuracy of the external timm leaderboard classifier on the generated images (Gen. Top5) drops moderately from 87.48% (in Table 5 of the manuscript) to 84.19%, primarily due to the challenge of generating samples for unseen classes. However, despite this constraint, our approach still outperforms all peer methods across concept preservation and text-image alignment metrics, demonstrating its superior ability to extrapolate fine-grained visual concepts beyond the training distribution. This aligns with findings from our long-tail and OOD experiments in Section 4.2, where our model demonstrated strong performance for underrepresented/unseen categories.\n\n*[1] IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models, Ye et al.*\n\n*[2] ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation, Wei et al.*"}]

Author rebuttal:

Meta-review: The paper proposes a generative data augmentation method that balances fidelity and diversity when generating synthetic training images from image and text prompts. The approach first trains a domain-specific embedding model to extract salient class features from individual images, then fine-tunes a diffusion model conditioned on these embeddings. This allows for generating diverse images while preserving input concept fidelity.

Reviewers noted strengths, including strong empirical results, especially in long-tail settings, a comprehensive evaluation across eight datasets, and clear presentation. Main weaknesses include computational costs from dataset-specific training and modest gains in standard settings. The rebuttal provided additional experiments on FGVC and scene data, a computational cost analysis, and more baseline comparisons. All reviewers found the responses satisfactory and recommend acceptance.

The Area Chair agrees with the reviewers and recommends acceptance.
Decision: Accept (poster)