Diversity-Enhanced and Classification-Aware Prompt Learning for Few-Shot Learning via Stable Diffusion
Authors: Gaoqin Chang
Jun Shu
Xiang Yuan
Deyu Meng
Venue: Accepted by TMLR ()

Abstract: Recent text-to-image generative models have exhibited an impressive ability to generate fairly realistic images from some text prompts. In this work, we explore to leverage off-the-shelf text-to-image generative models to train non-specific downstream few-shot classification model architectures using synthetic dataset to classify real images. Current approaches use hand-crafted or model-generated text prompts of text-to-image generative models to generate desired synthetic images, however, they have limited capability of generating diverse images. Especially, their synthetic datasets have relatively limited relevance to the downstream classification tasks. This makes them fairly hard to guarantee training models from synthetic images are efficient in practice. To address this issue, we propose a method capable of adaptively learning proper text prompts for the off-the-shelf diffusion model to generate diverse and classification-aware synthetic images. Our approach shows consistently improvements in various classification datasets, with results comparable to existing prompt designing methods. We find that replacing data generation strategy of existing zero/few-shot methods with proposed method could consistently improve downstream classification performance across different network architectures, demonstrating its model-agnostic potential for few-shot learning. This makes it possible to train an efficient downstream few-shot learning model from synthetic images generated by proposed method for real problems.

Published in Transactions on Machine Learning Research (09/2025)
Diversity-Enhanced and Classification-Aware Prompt Learn- ing for Few-Shot Learning via Stable Diffusion
changgaogin com School of Mathematics and Statistics, Ministry of EducationKey Lab of Intelligent Networks and Network Security, Jaotong University
Gaoqin Chang
junshu@mail.zzja edu.cn School Mathemalics and Statistics, Ministry of EducationKey Lab Intelligent Networks and Network Security, Xi an Jinotong University
Jun Shu"
Pengcheng Laboratory
Xiang Yuan
relojeffrey@gmmail School of Mathematics and Statistics, Ménistry of EducationKey Lab of Intelligent Networks and Network Security, Xi Jiotong University
Deyu Meng
dymeng@moil..gg edu.cn School of Mathematics and Statistics, Ministry of EducationKey Lab of Intelligent Networks and Network Security, Xi an Jinotong University Pengcheng Laboratory
Reviewed on OpenReview: https: //tp://oppee openreview. net/ forum? id=4CFliohpt
Abstract
Recent text-to-image generative models have exhibited an impressive ability to generate fairly realistic images from some text prompts. In this work, we explore to leverage off-the- shelf text-to-image generative models to train non-specific downstream few-shot classifica- tion model architectures using synthetic dataset to classify real images. Current approaches use hand-crafted or model-generated lext prompts of texl-lo-image generative models to generate desired synthetic images, however, they have limited capability of generating di- verse images. Especially, their synthetic datasets have relatively limited relevance to the downstream classification tasks. This makes them fairly hard to guarantee training models from synthetic images are efficient in practice. To address this issue, we propose a method capable of adaptively learning proper text prompts for the off-the-shelf diffusion model to generate diverse and classffcation-awaae synthetic images. Our approach shows consistently improvements in various classification datasets, with results comparable to existing prompt designing mothods. We find that replacing data generation strategy of existing pero/few. shot methods with proposed method could consistently improve downstream classification performance across different network architectures, demonstrating its model-agnostic poten- tial for few-shot learning. This makes it possible to train an efficient downstream few-shot learning model from synthetic images generated by proposed method for real problems. Code please see: https://gitae wtt.ccaoo.caeeccc htom/ccanngiiaoinn/ee....
Corresponding Author

---

Published in Transactions on Machine Learning Research (09/2025)
[a] Hand-crafted prompts
[b) Model-generated prompts
Vanilla prompt
Multi-domain prompt
LE method
CiP method
good photo the (class)
blue ane le and camving doen A gruup of alclena flytng pee fald
Asmel plane
A photo of (class)
sculpture of (class
Aglene thain the
pointing of (cass)
postion
Aamal plans foasing waterr
Generated images lack rich content information
Generated images follow similar styles and backgrounds
(c) Diversity-enhancce and Classification-awarr prompts (DeCap)
Diversiity unhanced prompt pool (collect by hand-crafted model-gemerteed prompts)
photo
An abstract
An airplane
An airplanc
plane flys. sky
airplane flys
airplane
airplane
meadow
cloudy
Classification prompts leamiing process
Prompts sultable to downstream few shot task
An abstract photo
An airplane grazes in
An airplane flys under cloudy sky
airplane flys over old farmhouse
airpllane
meadow
Generated images clasifficaion-awarr suitable to the airplane class
The comparion between existing prompt design methods and proposed DeCap method. Hand- crafted methods usually generate images with different domain information but limited content information. Model-generated methods overcome this shorteoming, while may generate images share similar patterns. DeCap constructs diversity-enhanced prompt pool that contains potentially all-inclusive prompt infor- mation by integrating the advantages hand-crafted and model-generated methods, and then carry out clasiffiattinn-wart prompt learning process mine proper prompts suitable to few-shot tasks from the prompt pool. Figure shows the mined prompts for airplane classification.
Figure 1:
1 Introduction
Recently, deep learning powered by large-scale annotated data has achieved great success the field of image recognition (He 2016). However, acquiring and curating a large-scale high-quality dataset can be notoriously costly and time-consuming. This is especially challenging for inherently expensive domains, such medical imaging, remote sensing, etc. Few-shot learning addresscs the data issue by training model using few data from the concerned tasks (Wang el. all, 2020; Song el all, 2023; Shu et all, 2018). Generally, few-shot learning models use specialised algorithms and architectures to achieve the objective (Zhang et all, 20231; Sendera all, 2123; Zhon el all, 2023a; Baik et all, 2023; Zhang et all, 2023: Shu et 20230 This limits the variety of model architectures and potential applicability for real-world problems.
To address this limitation, some rescarches focus on generating images which are then used to train a classification model. It has been early explored based on GANs models (Besnier el. all, 2021, Zhang all, 2021: labaniaa et all 2021), and become a research hotspot recently due to the rise of text-to-ine to-iiaaee diffusion models (Ho ct all 2020: Song & Ermon 20MM). Though these methods improve few-shot performance, some studies have found that synthetic datasets often have significant differences from real datasets and simply increasing the number of generated images cannot narrow the gap (Sariyildiz el al], 2023: He el all 2022).
Fortunately, some researchers (Bansal Grover, 2023; Shin et all 2023: Burg et all 2023a) explore generate high-quality images for improving the results, and potential path is to design proper text de- scriptions (prompts) for text-to-image generation diffusion models to generate desired synthetic images. A
2

---

Published in Transactions on Machine Learning Research (09/2025)
direct approach is to construct prompts by formatting class labels according to template (called vanilla prompt (Sariyildiz et all, 2023; Radior et all, 202T))), such "a photo of {class}". To produce more diverse text descriptions, multi-domain prompt (Shipard et all 2π²X) additionally provides list of domains with the prompt, c.g., " a {domain} of a {class}", to construct a set of prompt templates, in which '[domain)' refers to drawing, painting, sketch, elc. However, these hand-crafted prompts have limited capacity of gen- erating images with rich content information, which usually leads to inferior generalization performance when training downstream models. To improve the content quality of prompts, the language enhancement (LE) method (He et all, 2022) leverages an off-the-shelf word-to-sentence T5 model (lattel et all, 2020) to automatically expand class names into various sentences with rich content descriptions, containing the class names as language prompts. While this method hardly considers the class-relevant visual information for image classification. The CiP method (Lee el all 202)) generates high-quality prompts via extracting meaningful captions from real images using the off shelff image captioning models, showing a significant improvement generating informative synthetic images.
Although prompls produced by off-the-shelf foundational models can help generate high-quality images, they still have evident deficiencies in practice. On the one hand, generated prompts tend to share fixed or similar patterns for different images as reported in Wang et al. (2n2za), which may limit diversity of synthetic images. For example, as shown in Figure M. images generated by LE and CiP methods usually follow the similar styles and backgrounds. This limitation, which is even more serious under few-shot setting, may cause subpopulation shift problem (Nagarajan et all 22120 Yang et all 2123a), i.e., some subpopulations of synthetic images shift from real-world datasets. On the other hand, existing prompt designing methods have relatively limited relevance to the downstream classification tasks. Generally, the generated text prompts only employ class names or class-rolevant visual information, which lcads to some noises in generated prompts, e.g., prompts containing noisy labels or additional negative class information (please also see Figure E). Therefore, it is relatively hard to guarantee that training models from synthetic images are efficient for downstream classification tasks, which tends to hinder their application effectiveness and reduce their performance stability in real problems.
To alleviate the aforementioned issues, this paper presents a enhaaned and Classification-awarr prompt (DcCap) learning strategy to mine proper text prompts for downstream few-shot classification tasks (see Figure M for illustration). Our main idea is to build a diversity prompt pool, and then select suitable prompts from it. In detail, we first combine existing hand-crafted diverse prompt templates and rich content prompt descriptions generated by off-the-shelf foundational models to construct a prompt pool containing potentially all-inclusive diverse prompt information. And then we propose a novel meta-lcarning approach to select proper prompts tailored for the few-shot learning task from the prompt pool. The DeCap method involves two nested learning loops: an inner-loop to train a classification model using generated synthetic images, and an outer-loop to search suitable prompts for text-to-image foundational generative models that produce synthetic training images for the inner-level classification model. The given real few-shot images are employed to compute outer-loop meta-objective for helping achieve classification-wwree prompt learning. Through iteratively ameliorating both prompts selection and classification model performance, our method is capable of mining proper prompts which are attained specifically suitable to concerned few-shot task.
In summary, we make the following four-fold contributions:
(1) We propose an approach to automatically learn proper text prompts for texl-to-image diffussin models, enabling the generation of diverse, classification-wware synthetic images for few-shot learning tasks in meta-learning manner.
(2) We demonstrate that proposed DeCap method could not only automatically discover suitable prompts tailored to the concerned tasks (e.g., Section B but also adatively filter out noisy and low-quality
prompts which are potentially detrimental to classification model (e.g., Fig. B), by leveraging the proposed classification awareness meta-objective.
(3) We observe that diverse visual effects and content styles of mined prompts are potentially beneficial for producing high-quality training images tailored to specific tasks (e.g., Table B and Appendix DD), leading to consistent performance improvements across various datasets, as shown in Table M In contrast, existing
3

---

Published in Transactions on Machine Learning Research (09/2025)
methods often excel specific datasets and relatively fixed prompts but struggle with broader adaptability due to their limited prompt flexibility.
(4) We show that substituting the data generation strategy of existing zero prooffe-shot methods with our approach could further improve their performance across different algorithms and network architectures as shown in Table
The paper is organized as follows.Section 2 shows the related work. Section presents the proposed method. Section 4 demonstrates experimental results and the conclusion is finally made.
2 Related Work
Texa- Diffusion Model. Diffusion model etl all 2020 Song & Ermon, 201 has emerged as a research hotspot in the field of image generation recently, due to their impressive generative capabilities. It achieves gradual matching from a Gaussian distribution to an image distribution by reversing the diffusion
process. Recently, thanks large-scale image-lext paired datasets (Schuhmann et all, 2022) and the maturity of text-image foundation models such as CLIP, some stato-of-thh-art text-to-image diffusion models, including
2022), GLIDE (Nichol el. all, 2122); Imagen (Sabaria el all, 2022), and Stable DALL-E (Ramesh all
Diffusion (Rombach et al, 2T22). can produce wide variety of highly realistic images, which has greatly propelled rescarch in fields such as art (Wahid et all, 2023), style transfer (Yyan ct all, 2022 [Zhang et all, 20283), image controlling (Ruuz el all, 2028; Zhang et all, 20286 Gal et all, 2022), image editing (Bhnathraa aLl 2020, Bodur el all ()) data augmentation (Irabucco el all, 2I2K: Dunlap el all 20224 etc. Tn this paper, we explore leveraging off-the-shelf diffusion models to generate high-quality synthetic images for downstream few-shot image recognition.
research(Besnier Zhang et al 20ZI Uahamian et all (0zI) explored the role of synthetic datasets with GAN models, but their application scenarios are constrained due to the limited capabilities of carly GANs. With the emergence of large-scale text-to-image generative models, recent studies have validated the utility of synthetic datasets at a large scale. For example, for classification tasks, Sariyildiz et all (20123); Bansal L Groved (21128 train
Synthetic Dataset. for Image Recognition. In the early stages, some research (Besnter et all, 2021:
synthetic ImageNet datasets from scratch, He (2022)) D Li all (2022) showing that CLIP (Badlor
all 202T) can boost performance from synthetic datasets. Tian all (20ze1) validates the outstanding
performance of synthetic dataset using SimCLR and MAE models. In the field of object detection, Karazija
et all (20235) utilizes the output results of generative model's cross-attcntion layers as weak supervision for zero-shot object recognition. Additionally, synthetic datasets are also applied to addressing long-tail problems (Shin et all 2127)
The data generation stratogy could be roughly divided into two categorics. One is fine-tuning based mothod (Azizi all, 2023; Yuan el all, 2123), which fine-lunes generalive models' parameters using task data. These methods demonstrate strong domain adaptation capabilities on large-scale datasets and can effectively generate samples that conform to the distribution of real datasct. However, it often requires large-scale real datasets. Therefore, the other is prompt designing method to address few-shot learning. They don't alter the parameters of generative models; instead, it focuses on setting proper prompts for off-the-shelf generative models to generate synthetic datasets. As discussed in Section I1 there exist two methodologies of setting prompts, i.c., hand-crafted and model-generated prompts. While they are not sufficient to generate high- quality images for classification. In this paper, we propose to integrate the advantages of both methodologies to achiove a diversity-enhanced and classification-awaa prompt learning strategy. We need to clarify that, different from prompt learning methods (Zhoou all (I specifically designed for multimodal models like CLIP, which directly helps adjust focuses on generating efficient synthetic data for further help train downstream few-shot learning.
models prediction adapting to the concerned data, we
Meta Learning. Meta learning (Hospedalss el all, 2121: Shurel all 2026)) also known as learning to learn, focuses on how to quickly adapt and apply previously acquired knowledge when faced with new learning tasks.
Meta learning is widely used few-shot learning (Finn ef all 2017; Shn of all, 201S; Ravi Larochello, 2017
Snell all. 2017), hyperparameter optimization (Franceschi al 2018), transfer learning (Jang et all, 2014
Sun et all 2020) label noise learning (Shu et. all, 2019 21231; Wi all (02T), machine learning automation
4

---

Published in Transactions on Machine Learning Research (09/2025)
tmera Curcoy Droat)
filter low-quality/
Inner optimization
noisy prompts
mined prompts
Few-shot
prompt pool
outer optimization
Compute
good prompts
1 (o))
Downstream
training
Off-the-shell Stable Diffusion Model g(0)
Classification fwr(
Synthotic Dataset 1
quality
prompts
Figure 2: Overview of the proposed DeCap method. DeCap training involves two nested training loops. In the inner-loop optimization, we use the selected prompts set O to generate synthetic dataset and then help train the classification model, while in the outer-loop optimization, we search proper prompts attained specifically suitable to few-shot learning task from pre-constructed prompt pool.
X1I of (022)) etc. For image generation field, meta learning is used to achieve data distillation (Nguyen at 2020: Wang et al, 2018 20233 Stch 2020), data augmentation (Yamaguchi et all 2iZai), etc.
Different from previous works updating parameters of generative model, we use meta learning to set proper text prompts for diffusion models to generate high-quality images for concerned few-shot learning task.
3
The Proposed DeCap Method
3.1
Preliminary
For a N-classification task, We use ( = g(0,j, (x) to denote the generated image a! via an off-the- shelf text-to-image foundational models g, where E [N], [N] = [1,.... N} represents the i-th class, j E [M], [M] = (1,... ..,,,, where M means how many different prompts for this class, Oig represents the prompt used to generate this image, Ex, = 1,2, ... represents random gaussian noise, where means the generation number of each prompt. We denote the synthetic images generated by the prompts Oiy as X.y = [ (k) fk=1' We only study prompt sctting for image gencration, and we will drop explicit dependence of X on d for brevity in the following, i.e., X = g(0,)). Our approach can directly applied to different diffussin models, and we study the well-known open open-sourced Stable Diffusion (SD) model in this work.
Considering few-shot classification task with real data Dreat {(
1,..., .N..,N, 1,... K). where Tay. Mi denote image and its label, and N. K denote the number of classes and real images of each class, respectively. To boost few-shot model performance, it could use SD model to help generate high-quality synthetic data for few-shot image recognition tasks. Specifically, the synthetic data could be formulated as
syn
(0) {0,, i [N]], 0, [011, 02, ... 10.M.
X = {x g(0 [N], [M]].
For simplicity, denote the synthetic dataset D (0) {X*.mm (0), where Y (Bg, E [N], e [M]. Based on Day (f), we could train classification network fw by optimizing the following objective:
" arg min Clask (Sur, 5m (0)),
wEW
where W denotes parameter space, Ltask (Sw. Dewm NN Σn En E Closk (Ju( and glack de- note the cross-entropy loss for the few-shot classification task.
As discussed in Section existing prompt design methods may produce a limited diversity of synthetic im- ages, which can degrade the generalization performance of downstream classification models. Especially,
5

---

Published in Transactions on Machine Learning Research (09/2025)
could see that the prompt construction process of existing methods has limited relevance to the downstream classification tasks from (q)( i.e., it overlooks the explicit dependence of w" on 0. In other words, existing prompt learning methods are potentially classification-agnon -annosoiiin and limited flexibility, which greatly reduces the alignment between synthetic datascts and downstream classification task roquirement. To address these two issues, we propose a novel prompt learning stralegy called DeCap, which explores to learn proper prompts for generating high-quality images to improve downstream few-shot learning task. We present the method respectively.
and solving algorithm in Section K and 1
3.2 Proposed DeCap Method
The proposed DeCap method firstly constructs a diversity-enhancee prompt pool (Section 3.2.1) by integrat- ing the advantages of hand-crafted and model-generated methods, and then carry out classification-wwad prompt learning process (Section [К
to mine prompts suitable to downstream few-shot classification task.
3.2.1 Diversity- Enhanced Prompt Pool Construction
The prompts constructed by previous methods often foues on specific aspects (see Figure I). We proposed to integrate the ad advantages of both hand-crafted and model-generated methods to construct a prompt pool that contains potentially all-inclusive diverse prompt information. To some extent, such prompt pool cannot completely contain expected prompt information, while our experiments demonstrate that it is sufficient to help generate high-quality images for boost few-shot performance. More comprehensive pool construction strategy is left for future research.
Specifically, we construct the prompt pool θ containing hand-crafted prompts and model generated prompts, for every class in the dataset. For hand-crafted prompts, first select some common prompt. templates provided by [Radford all (202I)) which contain various domain information. Then we manually add some prompts into the pool, covering aspects such as color, style, camera angle and so on. Since these prompts describe the object in general terms, we share these prompts for all classes. For model generated prompts, We employed the LE method (He all, 2022), which uses only class labels as information, and the CiP method (Lei et all, 2023), which leverages real image information to produce image captions. These prompts describe the object in detail, so different classes will have totally different descriptions. In a nutshell, for each category's prompt 0,, it contains two parts: the hand-crafted prompt 0²t and the model-generated prompt 8TH i.c., 0, [0" 0,7',, where all classes share the same template 0 while possess private prompt 0.""
After conducting this process, there already exists potentially adequate prompts containing both diverse domain and content information in the prompt pool. We give a simple example about what our prompt pool looks like in Appendix A. However, this prompt pool is overly abundant and clasification-agnostii in other words, the pool contains not only proper prompts but also noisy prompts for specific few-shot learning task. An illustration of the necessity of prompt learning process please see Appendix CAl Therefore, we further propose classification-waa. prompt learning strategy mine suitable prompts tailored to the concerned few-shot task form the prompt pool in meta-learning manner to help generate high-quality images.
3.2.2 ClasifficationAAarre Prompt Learning
To establish the connection between prompt setting process and downstream classification model learning, natural idea is to directly generate images approximating real few-shot image domain, e.g., SDEdit (Meng et 202T) and Textual inversion (Gal et all 212z). However, these methods tend to generate images that are very similar to real few-shot data, which is prone to overfitting, making it hard to generalize to unseen images (Please see Appendix CK and Table HI.. To overcome this challenge, we explore to utilize real few-shot data at the higher level, i.e., evaluating the classification performance on these real data, rather than focusing on achieving visual similarity to the real data. To achieve this goal, we formulate the prompt learning as the following bi-level optimization objective:
0 = arg min L (fw-(o): Dreat), Aee
(2)
where w² w²( = arg min
Crask (fw, Daw De),
(3)
wEW
6

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm Learning Algorithm of the DeCap Method
Downstream few-shot learning task real dataset Dreat Algorithm iteration number max-iler, popu- lation quantity popsize; Constructed prompt pool pool; off-the-shell -axttt diffusion model Output: Optimal prompt set. 0
Input:
1:
2: for iler = 1, 2..,..., max-iler do
3:
4: for m =
do # An individual of population pop!m) =GA.sample()
# See Algorithm in Appendir
8:
((m). yau get (pompt(pop!"") pool)
(oun()(m) ((0(m))
10:
# Eq.
w' w²(0(m)) (Q(m)) = arg min Clask (fws Doyn (0(m))
11:
12:
# Eq. (33)
fit popti)) m (fw- (w-((tm)) (omm), Dreat)
13:
fit.append( (fil poptm)) Pop.append(pop(ny) 15: end for
14:
16:
GA..pdate(ftt,Pop)
17:
end for
18:
return G.A.best.
where the inner-level objective (E) the same as Eq.(M), while the outer-level meta-objective L is computed by evaluating the performance of obtained classification model Eq-(2) on real few-shot data Dreat which utilizes real data to guide the prompt learning at higher level. Note that existing methods predefine the prompts while overlook the explicit dependence of classification model w² on prompts O. As comparion, explicitly require the performance of classification model to depend on the prompts O. Specifically, given prompt set 0 E O, we use these prompts to obtain the synthetic dataset Daym p0), and then train the downstream classífication model w=( on the synthetic dataset. With such dependency, the classiiication-wwae meta-objective then help to search proper prompts from prompt pool, which aims to improve the classification performance on the real few-shot data.
3.3 Learning Algorithm of the DeCap Method
To solve above bi-level optimization objective, through iteratively ameliorating both searching prompts at outer-level learning and classification model performance at inner-level learning, our algorithm is capable of mining clasification--wwree prompts which is attained specifically suitable to downstream few-shot task.
In our implementation, the optimization of 0 E θ is actually a discrete prompt selection problem since θ represents a discrete prompt Ther.fher,, we use the genetic algorithm (GA) (Katoch all 202T) to solve the outer-level optimization objective in (Z). Generally speaking. a genetic algorithm first generates different inputs, then obtains the corresponding value function outputs for these inputs, adjusts the scarch direction basod on the magnitude of the outputs, and eventually completes the optimization process. Therefore, only need to define the GA's input and value funetion for DeCap objective, and then genetic algorithm can be employed to mine proper prompts O* from constructed prompt pool θ in Section 322.
In our implementation, the input of GA algorithm is defined as a vector of integers. The length of the vector represents the number of prompts selected, and each dimension the vector corresponds to the index of the selected prompt, with values ranging from to the size of the prompt pool. Under this definition, cach input represents a different combination of selected prompts. The value function is defined the outer-level meta-objective meta in Eq. (2). The whole learning algorithm of proposed DeCap method summarized in
7

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm M. More details about genetic algorithm please see Appendix A. Morcover, also discuss the selection of optimization methods and the implementation details for DeCap method at Appendix A.
4 Experimental Results
4.1 Few-Shot Classification Performance
We compared with existing prompt. designing strategies including: (1) vanilla prompt (Raddlrd Lt all, 2020): using the template "a photo of {class)". (2) multi-domain prompt: using different text templates from domains provided in Radiord al (202T). (3) LE (le all, 2022): using language models for text prompt construction, where the input and output of language models are the class label and a sentence containing the class label, respectively. (4) CiP ellal, 2023): generating captions for real image data using image caption model. Model selection details please see Appendix BM2A We conduct experiments on 12 datasets: CIFARIO, STL-10, Imagenette, Pets, Caltech-101, ImageNett100, EuroSAT, FGVC Aircraft, Country211, DTD, UCF101, Imagenet. Datasets details are introduced in Appendix B... For the selection of the classification model, we use the CLIP model, as it has shown powerful classification ability. The training strategy we used strictly follows the settings described in He (2n22), where we finetune CLIP with generated data. We use "a photo of {class}" the text initialization for CLIP tuning for all datasets to eliminate the impact of different initializations on the evaluation of each method. Training and evaluating details are presented in Appendix B2 Table WI shows the few-shot classification performance of cach method on nine downstream few-shot learning datasets, where "without real" means that we only use synthetic images to train downstream models, while "with real" means that we use both synthetic and real few-shot images. Some ablation studics and analyses on proposed DcCap method plcase see Appendix MB
Using synthetic data to train rewm classification model, all prompl designing methods can improve CLIP zero-shot performance, showing that generating synthetic data is helpful to boost downstream clas- sification model, and proposed DeCap method demonstrates the best classification accuracies across all datasets. Note that existing methods often use relatively fixed-pattern prompts, which exhibits limitations in adapting to diverse classification tasks. As for datasets with simple categories like STI.110, CIFAR-10 10 and Imagenette, the hand-crafted prompts could achieve superior performance than model-generated prompts, illustrating that the prompts with only tasks; while for datasets with complex categories like Pets, Caltech-101,, Imagenet Arngenett00,EurrS... and Country211 datasets, the model-generated prompts could achieve better performance than hand-crafted prompts, implying that rich content information is more helpful to address these complex tasks. These results reveal that effective prompts should be set considering the characteristic of concerned tasks.
information may be relatively more proper for these
Aireraft
To address this challenge, proposed DeCap method could adaptively Icarn proper prompts suitable to the concerned tasks by reconciling class/domal information and rich content information (visualization mined prompts see Appendix 12.2), so as to generate better training images. On the whole, our method achieves an average performance improvement of 1.30% point compared to the best results of existing method on different datasets. Other metrics evaluating few-shot performance including precision, recall and F1-score, are also reported in Appendix Cdl and our method delivers consistent. improvements. We also evaluate the adversarial robustness of these methods in Appendix Ma, which further substantiate the high-quality data generation capability of proposed DeCap method.
When using additional real data, CLIP's performance could be further improved, though the number of real data relatively smaller than synthotic data. This implics that the quality of real data may be higher than that of synthetic data. All prompt designing methods obtain a further improvement over only using synthetic data. Even so, DeCap method still shows advantages over other methods on the most datasets (only one dataset slightly lower than best method), demonstrating that our approach could genuinely generate high-quality images as a supplement to real few-shot data to further boost the classification performance. These experimental results substantiate the capability of proposed DeCap method in generating high-quality images for boosting few-shot classification performance.
Notice that the CiP method demonstrates strong performance on some tasks by leveraging real data in- formation, while it may be limited effectiveness in utilizing real data via approximating visual similarity.
8

---

Published in Transactions on Machine Learning Research (09/2025)
Table 1: Top-1 classification accuracy on different datasets. We randomly selected 10 images per class
construct the few-shot datasets. Bold scores represent the best results cach datasets, and the second best scores are marked by underline.
Imagenet1os 100
Imagenette
Callech- 101
Country211
STL-10
CIFAR1O
EuroSAT
Aireraft.
UCF101
Imageenet
DTD
Pets
without real
zero-shot
94,26
70.25
97.22
81.85
83.89
70.14
23.11
17.07
13.44
42.39
60.80
58.18
vanilla prompt
95.33
72.37
97.69
82.29
84.74
70.62
31.31
17.04
13.72
49.20
60.83
58.45
multi-domain
94.97
70.66
97.89
83.07
87.56
70.50
30.11
17.85
13.90
49.63
60.83
58.64
LE
94.61
70.33
97.45
83.24
84.03
70.73
29.35
17.73
14.14
52.50
61.06
G8.86
CIP
94.92
70.24
97.65
84.04
88.12
70.76
39.91
18.00
14.98
60.69
63.63
59.75
DeCap (ours)
95.91
76.98
97.95
85.36
88.67
71.08
41.94
19.74
15.44
62.39
64.45
60.29
with real
real-only
94.28
70.33
97
81,96
84.51
70.33
24.15
19.41
13.80
42.55
61.09
58.51
vanilla prompt
95.55
76.20
98.00
83.84
89.85
70.8T
47.83
18.21
13.77
51.97
60.85
59.12
multi-domain
95.02
74.54
97.92
84.56
90.31
70.62
43.30
18.99
13.90
51.76
60.90
59.17
LE
94.72
71.66
97.49
84.00
84.34
70.46
42.06
20.22
14.37
54.04
61.54
69.85
CiP
95.05
70.51
97.75
86.16
89.86
70.86
49.17
20.31
15.49
62.66
64.66
62.51
DeCap (ours)
95.93
77.19
98.03
85.78
89.87
71.11
50.22
20.64
15.68
63.99
65.21
62.82
As comparion, proposed DeCap method utilizes real data al higher meta-level to facilitate the trained classifier based on synthetic images to perform well on unseen real images. As a result, our method achieves consistently improve over CiP method. To further illustrate this point, we additionally compared two com- monly used image augmentation methods including SDEdit (Meng et all, 202T) and Textual inversion (Gal Btall 2022) the Appendix K42,, which tend to produce similar images based on given real few-shot images. In Appendix. [ann, we demonstrate that even synthetic images with significant difference with real images in terms of visual similarity (sce Figure H), can still bring much improvements to classification performance. These results further show that Decap method exhibits better flexibility and adaptability in prompt setting via utilizing real few-shot images at the meta-level.
4.2 Comparion with SOTA Methods
In Section I have shown that DeCap method performs better than other prompt designing methods under the same CLIP classification model. The key goal of DeCap method is to mine proper prompts to generate high-quality data for improving few-shot learning, while it is not confined to specialised algorithms and architectures to complete fow-shot lcarning tasks. To illustrate this, we explore to use synthetic data of DeCap method to evaluate its capacity in improving other zero/few-shol algorithms and architectures.
Specifically, we conducted our experiments on three SOTA algorithms: (1) Fakelt (Sariyildiz et all 2023): It uses synthetic datasets to train the network on ResNet-50. (2) SuS-X (Ldandarao of all, 20Z3): : It leverages synthetic datasets dynamic support set and extends Tip-Adapter by utilizing the image-text distance.
(3) CaFo (Zhang et all, 2023d): Tt augments few-shot datasets with synthetic data and then combines the predictions of pre-trained CLIP and DINO. In our implementations, we replaced the data generation strategics of these methods with DeCap without altering any of model architectures for a fair comparison, and follow original settings of these methods to train the corresponding classification models. We provide experimental details Appendix BE and more results Appendix [.G
Table A reports the results. Notice that FakeIt method uses synthetic data to train the ResNet-50 model from scratch, which eliminates the effect of pre-training data for downstream classification tasks. Thus the performance of trained ResNet-50 model could appropriately reflect the quality of synthetic data. The DeCap method achieves a significant improvement of 7.43% points over original data generation strategy of Fakelt, substantiating that our method classifieraagnostic and capable of generating high-quality data suitable to concerned tasks. Though SuS-X and CaFo methods use pro-trained models, synthotic data of DeCap method could still outperform these methods in the vast majority of datasets. These results demonstrate that synthetic data of our DeCap method are not confined to specialised algorithms and classifier architectures.
9

---

Published in Transactions on Machine Learning Research (09/2025)
dog plays
photo of monkey, cat sitting branch
STL10: a pixelated photo dog.
the and truck all open
photo of monkey, cat sitting the grass
CIFAR10: photo of the nice
Naisy Label Error
(b).Model Caption Error
Lo) Quality Prompts
photo of dog, dog is running in the snow.
photo of monkey, monkey eating berry tree
STL10: cartoon dog.
cartoon car
A white cat on blue couch
CIFAR10: white
(d).Mined Reliable Prompts By DeCap
Figure 3: Illustration of (a) noisy label, (b) model caption error or (c) low quality prompts in prompt pool generated by existing hand-crafted and model generated prompt designing methods, and (d) mined reliable prompts by our DeCap method.
Table 2: Comparion of DeCap and SOTA methods on different datasets. "Method + DeCap" denotes the
performance of replacing original synthetic data strategies of each method with proposed DeCap method without altering any of model architectures for fair comparison.
Imagenet 100
Country211
STI-10
CIFARIO
Caltech--10
AireraR
UCFI01
Imagenet
DTD
Pots
FakeIt
32.26
38.45
69.60
29.74
66.20
32.75
48.40
37.70
3.61
31.25
33.30
15.46
DeCap
60.39
48.80
75.40
55.22
70.51
39.21
51.20
40.60
4.22
41.67
41.07
21.59
SuS-X
72.77
84.57
12.97
46.65
57.84
With DeCap
95.43
75.89
98.39
80.40
84.89
70.3
37.37
13.63
56.49
59.54
CaFo
95.33
85.34
97.66
86.62
94.09
74.64
83.5
26.07
16.20
65.74
72.69
63.41
With DeCap
95.90
86.00
98.06
88.66
94.28
76.28
85.59
32.10
16.88
68.03
75.02
64.29
This implies that our DeCap method is algorithm-agnostic for improving downstream few-shot learning tasks, and hopeful to be readily applied to real-world problems and tasks.
4.3 Why Proposed DeCap Method Perform Better?
In this section, further present some analysis of DeCap method in two aspects: robustness against noisy or low quality prompts, and data value analysis of synthetic data.
4.3.1 Robustness Against Noisy Low Quality Prompts
Existing prompls methods may set prompts for specific classes, which are polentially noisy or low quality for downstream few-shot tasks. E.g., for LE method, may generate prompts that contain not only the class we want, but also other classes information in the datasct. An illustrated example presented Fig (a): for STL10 dataset, when we generate images for "dog" "dog""""ar" classes, some images also contain information of "cat" /Struck" classes. Since "cat." /ttruc"" classes belong to the dataset, these prompts would generate
10

---

Published in Transactions on Machine Learning Research (09/2025)
doodle of the car.
dog walking sunny day
photo of the bird.
deer green pond.
olympic athletes racing during racing match.
photo of dog dog is running in the embroidered bird.
a photo of the clean deer.
photo of dog, dog plays in water with stick
dynamic car.
art of the
a cartoon deer.
Table Examples of synthetic images generated by DeCap method for STL-10 dataset.
images with noisy labels for the classification of "dog"/"car" For CiP method, due to the limitations of the image caption model's capability, it cannot always accurately annolate images, which may result in misidentifiictiinss. Although CiP method recognizes this issue and employs a prompt concatenation method like "a photo of (class), fimage caption]" reduce the influence of noisy captions, we found this may not always work. For example, monkey cat, the defined prompt "a photo of monkey, cat sitting in branch" may generate image that blending features of cat and monkey. The issue of misidentiifiation is particularly prominent in certain tasks, such as CIFAR10, where the low resolution images significantly impact the model's judgments. This explains why the CiP method performs poorly on CIFAR10 dataset, as presented in Table MI Moreover, hand-crafted prompts often introduce different domain information to construct diverse prompts. Generally, only part of domain information is reliable, while an amount of domain information may be of low quality for the concerned tasks. As shown in Fig (c), though both of prompts could generate images of dog, the improper domain information could hinder the performance of concerned classification models, c.g., the synthetic pixelated images may provide low-quality training data for STL-10 task. In Appendix Kwn further illustrate influence of prompts with domain information on the synthetic images.
shown in Fig (b), when the image caption model mistakenly identifies
Unfortumately, these noisy prompts are relatively hard to be filtered using data clcaning strategies such as CLIP filtering (IH el. all, 2022). address the issue, proposed DeCap method aims to mine proper prompts suitable the concerned classification task in meta-learning manner. As shown Fig (d), with such higher-level downstream classiication-awaree outer-loop supervised information, DeCap method could adaptively select effective prompts that help boost downstream classification performance, and discard aforementioned potential noisy prompts that would potentially hurt downstream classification performance.
4.3.2 Data Value Analysis of Synthetic Data
To better analyze why DeCap method outperforms existing prompl. designing methods, we use "leave- one-out" method (Shoozbani Zou, 2014) to evaluate data valuation, and then select typical high-quality images generated by DeCap method. Specifically, given dataset and a measure function we use à = (i)) (D) to represent data valuation of the synthetic image i. In our implementation, we use the dataset generated by vanilla prompt method the benchmark dataset S and classífication accuracy
11

---

Published in Transactions on Machine Learning Research (09/202))
(a) CIFAR-10 dataset
(b) STL-10 dataset
Figure 4: Illustaation of the distributions of learned hand-crafted prompts by DeCap method on (a) CIFAR- 10 and (b) STL-10 datasets. Each column represents the same template, and each row indicates which prompts were selected for each class. Black indicates prompt is not selected, orange indicates the prompt is selected once, and while indicates the prompt is selected more than once. For clarity, we removed prompts that were never selected.
the measure function V. Then we could compute data valuation of synthetic images generated by DeCap method via adding one image at time. Table & visualizes the synthetic images with high data valuations for STI.-10 dataset, and more visualizations are shown in Appendix DJI. As it shown, we can see that synthetic images contain various patterns such as image style, background, camera angles, and actions, providing novel, diverse, and meaningful content information for given real few-shot data. This indicates that DeCap method does mine proper diverse and rich content prompts suitable to concerned few-shot learning tasks, naturally leading to its better accuracies than other prompt designing methods.
4.3.3 Visualization of Learned Prompts
Fig. and in Appendix D2 show that proposed method could adaptively adjust the proportions of model-generated and hand-crafted prompts, so to reconcile class/domain information and rich content information for different classes. The distributions of learned hand-crafted prompts are shown in Fig can find that domain information required by each class are distinctly different, which implies that proposed method could adaptively learning suitable prompts for different classes of specific few-shot tasks. Table Ii shows that consistently selected prompts are with high diversity and fine-grained information, including: movement, posture, background, color, quantity, other objects, and so on. More analysis please refer to Appendix DE These results substantiate the capability of DeCap method on adaptively mining suitable prompts tailored to the concerned lasks, naturally leading to enhancement of classification performance.
5
Conclusion
We present the DeCap, novel adaptive prompt learning approach to generate diverse and classification- aware synthetic data for ream few-shot learning in meta-learning manner. Proposed DeCap method could mine potential reliable prompts suitable to downstream few-shot learning tasks, demonstrating im- pressive capabilities in improving downstream classification models for different few-shot lcarning tasks compared with existing prompt designing mots We could further boost existing SOTA zero/ pero/fee-shot learning methods by simply replacing data generation strategy with the proposed method, showing its poten- tial model-agnostic characteristics. Besides, we also provide some intuitive visual interpretation, providing an initial insight into proposed DeCap method. Such an adaptive prompt learning approach based on clasification-awaree meta-objective is proming establish the connection between pixel-level data genera- tion and image-level semantic understanding, and hopeful to be employed to other computer vision tasks, like semantic segmentation and object detection, etc.
Impact Statement
In this work, we study the potential high-level computer vision applications of the popular diffusion models. We believe that only using the low-level visual similarity cannot effectively help complete high-level vision tasks. The dassificationnawarre meta-objective provide a novel perspective that establishes the connection between low-level vision and high-level vision, aiming to advance the field of machine learning and computer vision. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Acknowledgements
This work was supported by the National Key Research and Development Program of China (2022YF11004100; in part by the National Natural Science Foundation of China (62476214, 12326606, C24A20324 and 82430017) and the Tianyuan Fund for Mathematics of the National Natural Science Foun- dation of China (12426105), and part by the Major Key Project of Pengcheng Laboratory under Grant PCL2024A06.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXie:2303.00 087744 2023.
Shekoofeh Azizi, Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David J Fleet. Synthetic data from diffusion models improves imagenet classification. arXiv preprint arXiv:2304.08 08466 2023.
Sungyong Baik, Myungsub Choi, Janghoon Choi, Heewon Kim, and Kyoung Mu Lee. Learning to learn task- adaptive hyperparameters for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Hritik Bansal and Aditya Grover. Leaving reality to imagination: Robust classification generated datasets. In ICLR 2023 Workshop on Trushworthy and Reliable Large-Scale Machine Learning Models, 2023. URL
https://openrevieer httpe/vvieweee/ffrurreee wwtp.ooenefoood...........
Victor Besmier, Himalaya Jain, Andrei Bursuc, Matthieu Cord, and Patrick Pérez. This dataset does not exist: training models from generated images. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Specch and Signal Processing (ICASSP), pp. 1-5. IEEE, 2020.
Binod Bhattarai, Seungryul Back, Rumeysa Bodur, and Tae-Kyun Kim. Sampling strategies for gan synthetic data. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2020.
Rumeysa Bodur, Binod Bhattarai, and Tac-Kyun Kim. Prompt augmentation for self-supervised text-guided image manipulation. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 2299-8838, 2024a.
Rumeysa Bodur, Erhan Gundogdu, Binod Bhattarai, Tae-Kyun Kim, Michael Donoser, and Loris Baz- zani. iedit: Localised text-guided image editing with weak supervision. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp.
2024b.
Max F Burg, Florian Wenzel, Dominik Zietlow, Max Horn, Osama Makansi, Francesco Locatello, and Chris Russell. A data augmentation perspective on diffusion models and retrieval. arXiv, 2023a.
Max F Burg, Florian Wenzel, Dominik Zietlow, Max Horn, Osama Makansi, Francesco Locatello, and Chris Russell. Image retrieval outperforms diffusion models on data augmentation. Transactions on Machine Learning Research, 2023b.
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Fkin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference on computer vision and pallern recomnition, pp. 113 123, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largo-scale hicrarchical image database. In 2009 IEEE conference on computer vision and pottern recognition, pp. 248-255. Icce, 2009.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang. Joseph E Gonzalez, and Trevor Darrell. Diversify your vision datasets with automatic diffusion-based augmentation. Advances in Neural Information. Processing Systems, 36, 2024.
Li Fei-Fei, Robert Fergus, and Pietro Perona. One-shot learning of object categories. TEEE tronsactions pottern analysis and machine intelligence, 28(4)5594-11, 2006.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel pro- gramming for hyperparameter omiimization and mela-learning. In International conference on machine learning, pp. 1568 1577. PMLR, 2018.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint ar Xiv::2200.. 01618, 2022.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59):1-355 2016.
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, pp. 2242-2251. PMTR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn CVPR, 2016.
Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and Xiaojuan Qi. Is synthetic data from generative models ready for image recognition? arXiv preprint arXiv:2210.01574, 2022.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Jonathan Ho, Ajay Jain, and Picter Abbecl. Denoising diffusion probabilistic models. Aduances in neural
information processing systems, 33:6840 6851, 2020.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(9)55149-5169, 2021.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiu:1 2704.04861. 2017.
Jeremy Howard and Sylvain Gugger. Fastai: layered api for deep learning. Information, 11(2):108, 2020.
Khawar Islam, Muhammad Zaigham Zaheer, Arif Mahmood, and Karthik Nandakumar, Diffusemix: Label- preserving data augmentation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27621-27630, 2024.
Ali Jahanian, Xavier Puig, Yonglong Tian, and Phillip Isola. Generative models data source for multiview representation learning, arXiv preprint orXiv:2106.0522 2021.
Yunhun Jang, Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Learning what and where to transfer. International conference on machine learning, pp. 3030 3039. PMTRR, 2019.
14

---

Published in Transactions on Machine Learning Research (09/2025)
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference Computer Vision, pp. 709-727. Springer, 2022.
Laurynas Karazija, Tro Laina, Andrea Vedaldi, and Christian Rupprecht. Diffusion models for zero-shot open-vocabulary segmentation. arXiv preprint arXiv:2306. 09316, 2023a.
Laurynas Karazija, Iro Laina, Andrea Vedaldi, and Christian Rupprecht. Diffusion Models for Zero-Shot Open-Vocabulary Segmentation. arXiv preprint, 2023b.
Sourabh Katoch, Sumit Singh Chauhan, and Vijay Kumar. A review genetic algorithm: past, present, and future. Multimedia tools and applications, 80:8091 8126, 2021.
Hoki Kim. Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint orXiv:2010.01950,
2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arviv preprint 1XXic131226614, 2013.
Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, 2009.
Alexey Kurakin, lan J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRG, 2018.
Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332 1338, 2015. doi: 10.1126/ aa30.........03500 URL science. htt//do//abb/10.1125 www.ecc...aabaae...
https://www http://wwwwwcnncce.....
Shiye Lei, Hao Chen, Sen Zhang, Bo Zhao, and Dacheng Tao. Image captions are natural prompts for text-to-image models. arXiv preprint 037.:2307708566, 2023.
Bo Li, Haotian Liu, Liangyu Chen, Yong Jae Lee, Chunyuan Li, and Ziwei Liu. Benchmarking and analyzing generative data for visual recognition. arXiv preprint arXiv:2907.11 13697,, 2023.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain gener- alization. In Proceedings of the IEEE international conference computer vision, pp. 5542 5550, 2017.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-imagee pre-training for unified vision-language understanding and generation. In International conference on machine learning, pp. 22888-12900 PMLR, 2022.
Weixin Liang, Girmaw Abebe Tadesse, Daniel Ho, Li Fei-Fei, Matei Zaharia, Ce Zhang, and James Zou. Advances, challenges and opportunities in creating data for trustworthy ai. Nature Machine Intelligence, 4(8):669 677, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Represenlations, 2018.
S. Maji, Kannala, Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Frmon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint orXiv:2108.01073, 2021.
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of out-of-distributioon generalization. arXiv preprint arXiv:2010.1577 2020.
Timothy Nguyen, Zhourong Chen, and Jachoon Lee. Dataset meta-learning from kernel ridge-regression. arXiv preprint ar Xix::011.00003 2020.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishknn, Bob Megrew, Nlya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with texi- guided diffusion models. In JCML, 2022.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE IEEE, 2012.
conference computer vision and pattern recognition, pp.
Ed Pizzi, Sreya Dutta Roy, Sugosh Nagavara Ravindra, Priya Goyal, and Matthijs Douze. A self-supervised descriptor for image copy detection. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 14532- 14522, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Tiu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(14)): 67, 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. 2022.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In International conference on learning representations, 2016.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt,, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pp. 5389-5400. PMT.R, 2019.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Hightrosolutin image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognittin,, pp. 10684 10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500 22510, 2023.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorcalistic text- to-image diffusson models with deep language understanding. Advances in neural information processing 2022.
systems,
Mert Bulent Sariyildiz, Karteek Alahari, Diane Larlus, and Yannis Kalantidis. Fake it till you make it: Learning transferable representations from synthetic imagenet clones. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation.
arXiv preprinl ar Xrx:::2311.1000 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5h: An open large- scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 55:25278-25294 2022.
Marcin Sendera, Marcin Przwwiçllkkwssk, Konrad Karanowski, Macicj Zigba, Jacck Tabor, and Przemyslaw Spurek. Hypershot: Few-shot learning by kernel hypernetworks. In WACV, 2023.
Joonghyuk Shin, Minguk Kang, and Jacsik Park. Fill-up: Balancing long- tailed data with generative models. arXiv preprint ar Xiv:23000.00 07200, 2023.
16

---

Published in Transactions on Machine Learning Research (09/2025)
Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, and Clinton Fookes. Diversity is defi- nitely needed: Improving model-agnostic zero-shot classification via stable diffusion. In Proceedings of the IEEE/CW Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 769 778, June 2023.
Jun Shu, Zongben Xu, and Deyu Meng. Small sample learning in big data era. arXiv:1808. 0a0e:000080447,,, 2018.
arXiv preprinl
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meih-wcight-net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, 32, 2019.
Jun Shu, Deyu Meng, and Zongben Xu. Learning an explicit hyper-paramene prediction function conditioned on tasks. J. Mach. Learn. Res., 24:1 74, 2023a.
Jun Shu, Xiang Yuan, Deyu Meng, and Zongben Xu. Ciw-net: Learning a class-aware sample weighting mapping for robust deep learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023b.
Yang Shu, Zhangjie Cao, Jinghan Gao, Jianmin Wang, S Yu Philip, and Mingsheng Long. Omni-training: bridging pre-training and mela-training for few-shot learning. IEEE Transactions on Pullern Analysis and Machine Intelligence, 2023c.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arxiv preprinl Xre:: AXXi::1409.. 1409. 1556, 2014.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical works for few-shot learning. Adoances in neural information processing systems, 30, 2017.
Kihyuk Sohn, Huiwen Chang, José Lezama, Luisa Polania, Han Zhang, Yuan Hao, Irfan Essa, and Lu Jiang. Visual prompt tuning for generalive transfer learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognétion, pp. 19840-19855) 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yisheng Song, Ting Wang, Puyu Cai, Subrota. K. Mondal, and Jyoti Prakash Sahoo. comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities. ACM Computing Surveys, 55 (13s):1 40, 2023.
Felipe Petroski Such, Aditya Rawal, Jocl Lehman, Kenneth Stanley, and Jeffrey Clune. Generative tcaching networks: Accelerating neural architecture search by learning to generate synthetic training data. Tn International Conference on Machine Learning, pp. 9206 9216. PMLR, 2020.
Qianru Sun, Yaoyao Liu, Zhaozhong Chen, Tat-Seng Chua, and Bernt Schiele. Mcta-transfer learning through hard lasks. IEEE Transactions on Pallern Analysis and Machine Intelligence, 44(3):1443 14411144466,,,,, 2020.
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI 16, pp. 776 794. Springer, 2020.
Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. Stablerep: Synthetic images from text-to-image models make strong visual representation learners. Advances in Neural Information Processing Systems, 36, 2024.
Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakmutdinov. Effective data augmentation with diffusion models. Xin preprint or urXiv:2302.07944 2023.
17

---

Published in Transactions on Machine Learning Research (09/2025)
Vishaal Udandarao, Ankush Gupta, and Samuel Albanic. Sus-x: Training- free name-only transfer of vision- language models. In Proceedings of the IEEE/CVF International Conference. on Compuler Vision, pp. 2725 2736, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Collech-UCSD Birds-200- 200-2011 Dataset. 7 2011.
Risqo Wahid, Joel Mero, and Paavo Ritala. Written by chatgpt, illustrated by midjourney: generative ai for content marketing. Asia Pacific Journal of Marketing and Logistics, 35(8):1813 1822 2023.
Alex Jinpeng Wang. Kevin Qinghong Lin, David Junhao Zhang, Stan Weixian Lei, and Mike Zheng Shou. Too large; data reduction for vision-language pre-training, In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp.
2023a.
Boyu Wang, Jorge A Mendez, Changjaan Shui, Fan Zhou, Di Wu, Gezheng Xu, Christian Gagné, and Eric Eaton. Gap minimization for knowledge sharing and transfer. Journal of Machine Learning Research, 24 (33):1 57, 2023b.
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Newral Information Processing Systems, pp. 10506-10000 10518, 2019.
Kai Wang, Jianyang Gu, Daquan Zhou, Zheng Zhu, Wei Jiang, and Yang You. Dim: Distilling dataset into generative model. arXiv preprint ar NGv:2333.04707 2023c.
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexci A Efros. Dataset distillation. orXiv preprint arXiv:1811.10653 2018.
Yaqing Wang, Quanming Yao, James Kwok, and Lionel M Ni. Generalizing from a few examples: A survey on few-shot learning. ACM compuling surveys, 53(3):1 1-:4, 2020.
Zhicai Wang, Longhui Wei, Tan Wang, Heyu Chen, Yanbin Hao, Xiang Wang, Xiangnan He, and Qi Tian. Enhance image classification via inter-class image mixup with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 17223 17233, 2024.
Yichen Wu, Jun Shu, Qi Xic, Qian Zhao, and Deyu Meng. Learning to purify noisy labels via meta soft label corrector. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 10388 10396, 2021.
Zongben Xu, Jun Shu, and Deyu Meng. Simulating learning methodology (slem): an approach to machine learning automation. National Science Review, 18@e): 2024.
Shin'ya Yamaguchi, Daiki Chijiwa, Sekitoshi Kanai, Atsutoshi Kumagai, and Hisashi Kashima. Regulariz- ing neural networks with meta-learning generative models. Advonces in Newral Information Processing Systems, 36, 2024.
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change hard: A closer look at subpopulation shift. In ICML, 2023a.
Zuhao Yang, Fangneng Zhan, Kunhao Lin, Muyu Xu, and Shijian Lu. Ai-generated images data source: The dawn of synthetic cra. arXiv preprint arXiv:2310.018: 0183000,..0..0.0000111000 2023b.
Zhuoran Yu, Chenchen Zhu, Scan Culatana, Raghuraman Krishnamoorthi, Fanyi Xiao, and Yong Jac Lec.
Diversify, don't fine-tune: Scaling up visual recognition training with synthetic images. arXiv preprint Nii:2312.02253 2023.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Jianhao Yuan, Francesco Pinto, Adam Davies, and Philip Torr. Not just pretty pictures: Toward interven- tional data augmentation using text-to-image generators. arXiv preprinl XrXiv:2212. 11237, 2022.
Jianhao Yuan, Jie Zhang, Shuyang Sun, Philip Torr, and Bo Zhao. Real-fake: Effective training data synthesis through distribution matching. arXi preprint 2rX1::210.10044 2023.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint (tf::6600.0111 2016.
Baoquan Zhang, Xutao Li, Yunming Ye, and Shanshan Feng. Prototype completion for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023a.
Ji Zhang, Lianli Gao, Xu Luo, Hengtao Shen, and Jingkuan Song. Deta: Denoised task adaptation for few-shot learning. In ICCV, 2023b.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836 3847, 2023c.
Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Hanqiu Deng, Yu Qiao, Peng Gao, and Hongsheng Li. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot, learners. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 15211 15211-15222, 2023d.
Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion-based style transfer with diffusion models. In Proceedinggs of the JEEE/CU /VF conference on computer vision and pattern recognition, pp. 10146 1-44611666, 2023e.
Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean- Francois Lafleche, Adela Barriuso, Antonio Tor- ralba, and Sanja Fidler. Datasetgan: Efficient labeled data factory with minimal human effort. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 10145 10155, 2021.
Chenyu Zheng, Guoqiang Wu, and Chongxuan Li. Toward understanding generative data augmentation. Advances in neural information processing systems, 08:54046-54005 2023.
Fei Zhou, Peng Wang, Lei Zhang, Wei Wei, and Yanning Zhang. Revisiting prototypical network for cross domain few-shot learning. In CVPR, 2023a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16816-16825, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwci Liu. Learning to prompt for vision-language models. International Journal of Compuler Vision, 130(9):2337-233 2022b.
Yongchao Zhou, Hshmat Sahak, and Jimmy Ba. Training thin air: Improve image classification with generated data. ar Xiv preprint or ar X30:2300. 15316, 2023b.
19

---

Published in Transactions on Machine Learning Research (09/2025)
A More Details of Proposed DeCap Method
A.1 Discussion About the Optimization Process
We first explain why we don't use gradient-based methods like Finn et (2017); Sin all (20286);
Franceschi et all (201N) for optimization, and then describe our implementation details, which enable fast training.
One may think that in the outer loop (Eq 2). it is better to use gradient-based methods to learn continuous soft prompts like Dia et all (2022); Zhon ef all (20221); Sohn cf all (2123). However, the primary challenge lics in the computational cost. The parameters we want to update must be propagated through a large generalive model within the computation graph during backpropagation. This results an overwhelming computational burdeneven with 80GB of GPU memory, it is difficult to handle the computation for just few images. More critically, need to perform gradient optimization over the entire synthetic datasct, which requires backpropagation through thousands of images.
On the contrary, GA algorithm demonstrates clear advantages in our problem setting. On one hand, it discovers prompts with better interpretability as shown in See B:: on the other hand, it avoids the costly computation of meta-gradients and supports various nin-diffeeettialee operations.
In terms of implementation, we can leverage the strengths of the GA algorithm to store the images cor- responding to its prompt, thereby avoiding repeated image generation when prompt is selected multiple times. We also adopt the feature caching strategy proposed by HH (212z) to further accelerate train- ing. Additionally, since GA algorithm is suitable for parallel operation, the speed can be further accelerated by parallel computation. These implementation techniques significantly reduce the computational cost of DeCap algorithm: our code can run on a single GPU with 8GB of memory, and the optimization takes, on average, only 0.1 GPU hours per class. For example, on the STL10 dataset (10 classes), we consume about GPU hour, and for ImageNet100 (100 classos), roughly 10 GPU hours are needed.
A.2 Examples of Prompt Pool Construction
In this section, we give a simple example about what our prompt pool looks like.
Let us consider "cat v.s. dog' classification task. Assuming that our hand-crafted prompts are ["a photo of (",, "a sketch of (}., "a {} image"] and model-generated prompts are [cat:|"a cat on the grass", "a cute cat ",, dog:["a barking dog", "a dog in the room')] "] Then, our prompt pool will be:
[cat:[* (fa photo of fcal)", "a sketch of {cal)", "a {cat} image", fa cat on the grass', "a cute cat"), dog:["a photo of {dog)", "a sketch of {dog)", "a {dog} image", "a barking dog", "a dog in the room"]}
If we randomly select 2 prompts for each class, for example, the Oth and 3th prompts for cat, and 1th and 2th prompts for dog, which represents pop (0, 1, 1, the selected prompts for generating dataset will be {cat:[ "a photo of { "a cat on the grass"; ; dog: "a sketch of {dog}', "a {dog} image"]].
A.3 GA Algorithm Details
Genetic Algorithm (GA) is an optimization technique inspired by natural selection and genetic processes, widely used for complex combinatorial optimization problem-solving. Its key steps can be summarized as follows:
Initialization of Population: Randomly generate set number of individuals (solutions) to form the initial population, with each individual represented by gene encoding (typically binary string real numbers).
Fitness Evaluation: Assess the fitness of each individual using fitness function that quantifies their performance based the problems objectives.
20

---

Published in Transactions on Machine Learning Research (09/2025)
Selection:
Select individuals for the next generation based on their fitness values. Common selection methods include roulette wheel selection, tournament, selection, and rank selection, where fitter individuals have a higher chance of being chosen.
Crossover: Combine parts of two parent individuals' genes to produce new offspring, Crossover enhances genetic diversity, with methods like single-point, multi-point, and uniform crossover.
Introduce random changes to portion of individuals genes with certain probability, increasing genetic variation and helping to avoid local optima. Mutation can involve flipping gene bits or assigning random values.
Mutation:
Population Update: Merge the offspring with the current population and select suitable individuals based on fitness, often using elitism to retain the best solutions.
Termination Condition: Determine if termination criteria are met, such as reaching a maximum number of iterations, achieving predefined fitness goal, when improvements in fitness become negligible.
Present the final optimal solution or any satisfactory solutions, along with relevant analysis and validation.
Output Results:
Actually, in Algorithm M. the GA.update() operation mcans the steps from "selection' to "population update" operation. Our code are based on the scikit-opt library, and we use their default operators. What's more, unlike traditional meta learning meinn et all, 2017; Shu all 2018 Franceschi et 201; Jang at 2019: Sun et all 2021) relying on computing meta gradient to optimize outer-level meta loss, our outer-level optimization does not involve any meta gradient calculation (i.c., derivativo-free optimization), and we only execute gradient descent algorithm the inner-level optimization.
A.4 "Get_Prompp" Method in Algorithm II
Algorithm @ shows the "get prompt" method in Algorithm M We denote the number of classes as N, the name of these classes as "class__aaaae prompt numbers per class as M.
Algorithm 2 Get prompt Algorithm
Input: indexes pop, prompt pool pool; Output: prompt set: prompts, labels: Y
1: # pop is the inder of 0 = [0,, 02. ON T, RM in prompt pool
2: pop.reshape[N.
3: # Ysym contains every synthetic sample's 's label prompts=[] ysy=_ for 1. do
class-class namos(i] [ promppss.appenddggooll (laass[popp]]] 8: Ysum (i. (M))
0: end for 10: return prompts, "
B Implementation Details
B.1 Datasets Details
In this section, give brief introduction about datasets used in Section
21

---

Published in Transactions on Machine Learning Research (09/2025)
CIFAR10(Krizeevse 2009): The CIFAR10 dataset contains 10 common classes: airplane, car, bird, cat, dog, deer, frog, horse, ship, truck. Each class contains 6000 color images with 32 x 32 size. CIFARIO is widedly used in image classification.
STL-10 et all, 2011): The STL-10 dataset contains 10 common classes in real life: airplane, bird, car, cat, deer, dog, horse, monkey, ship, and truck. Although these photos comes from ImageNet, their annotations may be quite different, for example, "dog" class contains various dog breeds.
EnaagenetteHHowann & Gugger, 2020): Imagenette is subset of the larger ImageNet dataset, containing 10 easily distinguished classes: tench, English springer, cassette player, chain saw, church, French horn, garbage truck, gas pump, golf ball, parachute. It was created to provide a smaller, more manageable subset for training and lesting image classification models.
Pets (Parkii et ali, 2012): The Pets dataset consists of images of 12 different cats breeds and 25 different dogs breeds. It is commonly used for fino-grained classification tasks, where the goal is classify images into specific subcategories within broader class.
ImageNet100(Tian et all, 2020): ImageNet100 is subset of the original ImageNNtt dataset, containing 100 classes. It serves as a smaller alternative to the full ImageNet dataset for training and evaluating deep learning models for image classification tasks.
Caltech-101 (Fei-FeiFi et all, 2008): The Caltech-101 dataset is a widely used benchmark dataset for object recognition. It contains images of objects belonging to 101 distinct categories, including animals, vehicles, and household items.
EuroSAT (Helber et al, 2019): EuroSAT is a dataset of Sentinel-2 satellite images for land cover classi- 27,000 fication. It contains 27, ,000 RGB images across 10 classes, such as agriculture, forest, and water bodies, with
resolution of 64x64 pixels. It is widely used in remote sensing and euvironmental monitoring tasks.
Aircraft (Maji et all (0133: The FGVC Aireraft dataset is designed for fine-grained visual classification of aircraft. It includes 10,000 images of 102 different. aircraft models, focusing on distinguishing subtle differences between similar models. It is commonly used in fine-grained recognition research.
Country211 (Radford et all 2021)): Country211 is dataset released by OpenAI, designed to assess the geolocation capability of visual representations. It filters the YFCC100m dataset to find 211 countries that have at least 300 photos with GPS coordinates. OpenAl built a balanced dataset with 211 categories, by sampling 200 photos for training and 100 photos for testing, for each country.
B.2
Experimental Settings in Section 4
B.2.1 Model Selection
For the pre-trained generative model, we choose the Stable Diffusion XL-Turbo (SDXL-Turbo) model" for its fast generation speed and high quality image generation. This model takes text prompts as input and outputs images at a resolution of 512 x 512. During our experiments, we use ResNet-50 as the CLIP image encoder backbone. For classifier tuning (He el. all, 2022), different text prompt initializations may cause slight differences in accuracy, but since method focuses on the dataset quality, we simply use the vanilla template "a photo of {class}" for all the datascts.
To remonstrate that our method does not fail simply because stronger prompt generation models are used, in Tab.I we adopt different prompt generation models. For the LE method, we use the T5 model on CIFAR1O, STL-10, Imagenette, Pets, Caltech-101, and ImageNet100;; the GPT-3 results provided in (Bhatfara ef all (2020) on EuroSAT, FGVC Aircraft, and Country211; and the InternLM modef on DTD, JCF101, and ImageNet. For the CiP method, we use the BLIP model on CIFAR1O, STI-110, Imagenette, Pets, Caltech- 101, ImageNet100, EuroAAT, FGVC Aircraft, and Country211, while on the remaining three datasets we
22

---

Published in Transactions on Machine Learning Research (09/2025)
employ the Molmo model Although different datasets use different models, the results remain fair since both the baselines and our method adopt the same models.
B.2.2 Training Setting
Since Stable Diffusion XI-Turbo doesn't use classifier-free guidance, we simply set the guidance scale to 0 and we set inference steps to 2. For inner-level training of classification model, we generated 80 images for each class and trained for 20 epochs using the Adam optimizer with a learning rate from 2e - to 2e 5, equipped with the cosine learning rate schedule. For outer-level training, set the hyper-parameters of the GA algorithm as follows: popsize of 80, maxiter of 80. Regarding the selection of few-shot datasets, we randomly selected 10 images per class to construct the few-shot datasets. We do training on 8 NVIDIA A800 GPUs, with pytorch 1.12.1 and Ubuntu 20.04.
B.2.3 Evaluation Settings
The settings of stable diffusion model are the same in Appendix 12 We generated 800 images for each class and fine-tune CLIP for 30 epochs. We use the the AdamW optimizer equipped with the cosine learning schedule. After training, we use the fine-tuned CLIP model to do evaluation results are the average over 5 times run, with random seed in 7, 21, 42, 84, 105.
real test datasets. All the
B.3 Experimental Settings in Section
Fakelt (Sariyildiz et al., 2023): Fakelt use Stable Diffusion V1-4 model and different classifier-free guid- ance scale, but our generative model are not fit for using classifier-free guidance, re-implemented their generation approach under our generative model. Other training settings are the same with original paper, including classification model architecture, training learning rate, data augment strategy and so on.
SuS-X (Udandarao et all, 202x): The generative model of SuS-X is Stable Diffusion V1-4. For a better performance comparion, we reimplement SuS-X method with SDXL-Tuuro model for higher quality image generation. The prompt strategy and other experimental scttings keep the setting in the original paper.
CaFo et al, 2023)) Since CaFo utilizes the OpenAl model to generate description for CLIP text initialization, and the original model has been deprecated, we employed the simple template "a photo of fclass}" for text initialization across all datasets to ensure fairness. All other experimental settings remain consistent with the original paper. We have to point that CaFo is a few-shot learning method, and we only report the 16-shot result in Table 2 due to space limitation. Other shot results are given in Section Oii
C More Experimental Analysis of Proposed DeCap Method
C.1 Why Is Adaptive Prompt Learning Necessary?
To validate the necessity of adaptive prompt selection, we implement three prompt selection baseline strate- gies: (1) randomly selecting the same number of prompts from the prompt pool, denoted by Random. (2) using all prompts of the prompt pool, denoted by All. (3) randomly substitute half of DeCap prompts to randomly selecting prompts from th prompt pool, denoted by Half Random. Table shows the perfor- mance comparison on different datasets. All the experimental settings are the same as Appendix ( Haa We that the adaptive prompts selected by DeCap method could significantly improve classification model performance compared to random selection strategy. Besides, although using all prompts in the prompt pool offers more sulficient diversity than subset selection, it sulfers from various issues mentioned in Section [m which may deteriorates the performance of classification models. This explains that the performance of all prompts is only better than the random selection stratogy but not as good as DeCap method. To further illustrate whether or not the most contributive prompts are consistently selected, we randomly substitute half of the prompts learned by DeCap with randomly selecting prompts from prompt pool. It could be seen that when learned prompts are substituted, the performance would be impaired. That is say, the adaptive
https://hugringgface http:///ce...aaaooo///aaaaaaa/aa
23

---

Published in Transactions on Machine Learning Research (09/2025)
(a) Real
(b) SDEdit
(c) Textual Inversion
(d) DeCap
Figure 5: Examples syuthetic images by SDEdit, Textual Inversion, and DeCap methods. Compared with the other two methods, DeCap shows greater diversity and visual differences from real images, while DeCap method outperforms the other two methods in classification performance.
Table 4: Comparion of augmentation- based data generation methods and proposed DeCap method.
Methods
SDEdit
Textual inversion DeCap
STL10
94.74
94.86
95.90
Caltech-101
85.82
84.76
88.67
prompt learning process is necessary, and the mined prompts indeed contribute much classification perfor- mance improvements on the concerned few-shot lasks. These results further substantiate the capability of adaptively mining appropriate prompts for generating high-quality images for downstream few-shot learning tasks.
C.2 Compared with Data Augmentation Strategies Using Diffusion Model
Image augmentation methods based on diffusion models usually edit the given real images to generate novel synthetic images for training the classifier. In other words, these methods attempt to generate novel images based on visual similarity. Generally, prompt-based data generation methods mainly focus on how generate images when data are scarce or unavailable for the concerned task, while data augmentation methods focus on how further boost the performance when data are relatively enough. When the real images sparse, the synthetic images of data augmentation methods may lack diversity, which are prone to overfitting. In Table Hl we compared our method with two commonly used image augmentating ion moth on STL10 and Caltech-100 101 datasets: 1) SDEdit (Meng et al. 202T), using real image as the initial point of diffusion inverse process. 2) Textual inversion (Galet all 2022), learning the most similar prompts related to the given images. The results show that augmentation- based data generation methods achieve sub-optimal performance, while proposed DeCap method achieve significant performance improvements. This implies that compared with visual similarity guidance, our classification-awall meta-objective promising to set appropriate text prompts boost the performance of concerned few-shot tasks. Figure & shows some typical synthetic images generated by SDEdit, Textual Inversion, and DeCap methods. As it shown, proposed DeCap method could generate images with diversity and visual differences from real images, even some unseen "new" images. Please also see Appendix CM, our DeCap method could generate images that are visually distinct from real images while still maintain similar classification performance.
C.3 Ablation Study
We first conducted ablation experiments on two important parameters of our method: the number of prompts selected per class and the iteration count of the GA algorithm. The experimental results of the number of prompts selected per class are reported in Table We find that fewer prompts may lead to low datasct diversity, possibly hindering model performance, while more prompts may increase optimization difficulty, making hard to find the optimal solution. We suggest to set the number of prompts selected per class
24

---

Published in Transactions on Machine Learning Research (09/2025)
Table 5: Comparion of different random selection strategies with adaptive prompt selection strategy by
DeCap method.
STLI0
CIFARIO
Imagenette
Pets
Caltech-100 101
Imagenet100
EuroSAT
Aircraft
Country211
Random
94.74
71.94
97.25
82.26
85.76
70.48
36.76
17.88
14.56
All
95.19
74.41
97.33
82.45
84.45
70.66
37.26
18.57
14.32
Half Random
95.15
75.58
97.86
84.93
86.87
70.66
38.44
18.27
14.92
DeCap (Ours)
95.91
76.98
97.95
85.36
88.67
71.08
41.94
19.74
15.44
Table 6: Ablation study on the number of selecting prompts per class.
Table 7: Ablation study on the iteration number of GA algo- rithm.
Table Ablation study on extremely low shot real images case (1-shot).
1shot
10shot
vanilla.
CiP
10
20
40
20it
40it.
60it.
80it.
95.15
95.93
95.33
94.92
95.73
95.82
95.93
95.81
95.73
95.87
95.93
95.93
as 20. Table shows the performance of different iterations numbers of GA algorithm. We observed that performance of classification model converges around 80 generations. In our all experiments, we suggest to set the iterations number of GA algorithm as 80. Moreover, also report results of replacing the generative model from SDXL-Turbo to Playground-v2 in Tab.g We can see that our method still work when generative models are changed.
We further explore the result on extremely low-shot cases (e.g., 1-shot). We provide the results for the 1-shot scenario on the STL10 dataset in Table In extreme cases where real samples are severely lacking, our method may underperform compared to existing approaches due to the inability fully represent the task. Under such conditions, recommend leveraging the inherent knowledge of the generative model. We believe this result aligns with common sense: when there is large amount of data, we should make full use of real data. However, when data is extremely scarce, relying on the knowledge inherent in large models may be a better optionbecause the given data cannot fully represent the task. And we will explore to better address the issuc in the future work.
Table 9: The result. replacing the generative model from SDXI-Turbo to Playground-v2.
vanilla
multi
LE
CiP
DeCap
DTD
50.53
52.29
54.09
03.13
63.78
UCF101
60.89
60.91
61.35
64.26
64.84
C.4 Experimental Results on Different Evaluation Metrics
In this section, we give results other evaluation metrics including precision (Table mK),, recall (Table [ and Fl-sccore (Table 12), which are commonly used in few-shot learning, to further explore the robustness and gencralization ability of DeCap method. Some brief introduction about these metrics are given as follows:
Precision: Precision measures the accuracy of positive predictions. is defined as:
True Positives (TP)
Precision True Positives (TP) + False Positives (FP)
Precision answers the question: Of all the instances predicted as positive, how many are actually positive?
Recall: Recall, also known as sensitivity or true positive rate, measures the ability of the model to correctly identify positive instances. is defined as:
True Positives (TP)
Recall
True Positives (TP) + False Negatives (FN)
25

---

Published in Transactions on Machine Learning Research (09/2025)
Table 10: Precision results of different methods among all datasets.
vanilla
multi
LE
CiP
DeCap
STL10
95.35
94.90
94.17
95.16
95.63
CIFAR10
76.61
76.55
77.32
77.23
77.40
Im-10
97.27
97.30
97.27
97.30
97.34
Pets
82.58
84.76
84.17
84.52
85.70
Caltech-101
84.42
84.57
85.27
85.39
85.43
Im-100
69.82
71.27
69.28
73.03
71.90
EuroSAT
43.01
38.45
50.50
47.32
49.36
Aircraft
18.38
18.85
20.85
18.67
20.85
Country211
17.20
17.26
18.10
17.12
17.77
Table 11: Recall results of different methods among all datasets.
vanilla
multi
LE
CiP
DeCap
STL10
95.31
94.78
94.04
94.72
95.57
CIFAR10
72.39
69.63
68.24
68.63
76.99
Imagenette
97.25
97.28
97.25
97.28
97.33
Pets
81.69
82.05
82.13
83.12
84.52
Caltech-101
85.21
86.02
85.37
86.07
86.54
Imagenet 100
68.32
69.98
67.00
69.56
70.94
EuroSAT
31.07
29.62
28.31
40.62
42.22
Aircraft
17.03
17.84
17.72
17.98
19.71
Country211
13.72
13.90
14.14
14.98
15.44
Recall answers the question: Of all the actual positive instances, how many were correctly predicted??
F1 Score: The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both. It is defined as:
Precision Recall
F1 2
Precision + Recall
The Fi-score particularly useful when the class distribution uneven when precision and recall equally important.
The results demonstrate that our method performs well on these metrics, indicating that it not only achieves high accuracy but also excels in identifying positive samples and is more cautious when dealing with them. It more comprehensively illustrates the robustness and generalization of our method.
C.5 Adversarial Robustness
Adversarial learning aims to evaluate model robustness by adding small perturbations to the input data, causing the model make false predictions but making little difference human observers. We use two common attack methods: BIM (Basic Iterative Method) attack (Kuuakkn all, 201S) and PGD (Projccted Gradient Descent) attack (Madry al, MUN). The BIM employs an iterative gradient ascent approach, where at each step, BIM perturbs the image along the gradient direction predicted by the model. It be written as Zi+1 = + + VV,,, JoCz, where denotes the original image, denotes its label, and J means the gradient of loss function w.r.t. Zg. PGD further projects the adversarial examples into an e-ball around the original image.
We use classification model weights obtained from Scction P and implement adversarial attack on Ima- geNet100 validation dataset. We use torchattacks Kim (2027) library to conduct this experiment. We select attack step size 1/255 for these methods, and Fig reports the attack result. different attack
26

---

Published in Transactions on Machine Learning Research (09/2025)
Table 12: Fl-score results of different methods among all datasets.
vanilla
multi
LE
CiP
DeCap
STL10
95.29
94.71
93.99
94.74
95.58
CIFARIO
71.89
69.04
67.30
69.19
76.89
Imagenette
97.23
97.26
97.23
97.26
97.31
Pets
81.28
81.96
82.09
83.23
84.67
Caltech-
82.04
82.87
82.12
83.45
83.73
Imagenet 100
67.06
68.99
65.52
69.42
70.25
EuroSAT
26.36
24.43
24.38
36.36
39.34
Aireraft
15.10
15.98
16.00
15.95
17.81
Country211 13.18
13.35
13.62
14.36
14.93
PGD method Top-1 accuracy
BIM method Top-1 uccuracy
0.41
DeCap(ours)
DeCap(ours)
0.40
0.40
LE
0.39
LE
accuracy
0.39
Multi prompt
accurayy
Multi prompt
0.38
0.38
Vanilla prompt
Vanilla prompl
0.37
0.37
0.36
0.36
0.35
0.35
8
10
12
16
20
8
10
12
16
20
steps
steps
(a) PGD attack result
(b) BIM attack result
Figure Adersarial robustness of classification models trained with synthetic images using different prompts designing methods. We report results ImageNet100 validation set under two adversarial attack methods. The horizontal axis represents the number of steps taken in the attack, and the vertical axis represents the accuracy of the trained classification models on the validation set after the allack.
steps. We found that model-ggnerated prompts, due to their rich content details, have slight advantage in adversarial robustness compared to hand-crafted prompts. Moreover, since DeCap integrates the strengths of hand-crafted and model-ggenratted prompts methods, it consistently performs well in terms of resilience against adversarial attacks.
C.6 More Experimental Results of Section
In Scction 2 due to space limitations, we only present the compared results of the original algorithms and DeCap method. In Table IK, we additionally provide the compared results of other prompt design methods from Table on the SuS-X and CaFo algorithms. The experimental results are aligned with conclusions Section 2
Fig. / shows more experimental results different shots of each class for CaFo and CaFo + DeCap Methods. The experimental results are also aligned with conclusions in Section B.2.
C.7 Are Prompts with Domain Information Enough for Classification?
To illustrate this point, we conducted experiments on the Sketch subclass of the PACS dataset cf (2017), and the experiment settings are the same in Section B. For this dataset, all images follow the
27

---

Published in Transactions on Machine Learning Research (09/2025)
56
96.0
95.5
97.8
12
97.6
76
Cazo DeCop
94.5
Cafe DeCap
97.2
CaFo + DeCap
93.5
ADWSS
stots
shuts
(a) CIFARIO
(b) STL10
(c)
Imagenette
Cafo+ DeCap
66
CaFo
74
92
91
Cafo DeClp
Cafo DeCap
Sto0s
shots
thots
(d) Pets
(c) Caltech-101
(f)
Imagenet
CaFo DeCap
DeCap
30
Callo
GaFo
16.0
15.0
Caño DeCop Cafo
80
13.
16
16
2
abote
abots
chots
EuroSAT
(h) Aircraft
(i) Country211
Figure 7: Classification accuracics on different shots of each class for CaFo and CaFo + DeCap Methods
Table 13: More experimental results of SuS-X and CaFo methods. This table presents the classification accuracies of other data generation algorithms on these two fewo/tt methods. DeCap still achieves lcading performance the vast majority of datascts.
STL10 CIFAR10 Imagenette
Pets
Caltech-101
Imagenet100
EuroSAT
Aircraft
Country211
SuS-X
Multi 95.29
72.28
98.21
78.99
84.89
70.08
31.43
18.45
13.09
LE 95.39
73.62
98.37
79.20
$4.51
69.98
27.41
18.33
13.01
CiP 95.21
73.19
98.31
79.77
81.76
70.14
33.02
19.47
13.63
DeCap
95.43
75.89
98.39
80.40
84.89
70.30
37.37
19.83
13.63
CaFo
Multi
95.48
85.76
97.88
86.92
94.27
75.58
85.19
31.71
16.65
LE
95.51
86.00
97.78
88,06
94.03
75.00
85.63
26.37
15.27
CiP
95.33
84.05
97.88
87.38
94.27
75.92
85.64
27.18
14.83
DeCap
95.90
86.00
98.06
88.66
94.28
76.28
85.59
32.10
16.88
28

---

Published in Transactions on Machine Learning Research (09/2025)
Real data
sketch figure of guitar
DeCap prompt
skacc Paore of
$1.75
83
75.19
Figure 8: (Left) Examples of real images, synthetic images by hand-crafted prompts and DeCap methods on PACS Sketch datasct. (Right) Performance comparison between prior hand-crafted prompts and DeCap methods. Although DeCap appears visually distinct from real data, its classification performance can ap- proach that of training on the full dataset and significantly outperforms the prior prompts "a sketch figure of O'.
same style. As shown in Fig.8, although using the prior information "a sketch figure of yields better results than random domain information, while synthetic images by DeCap method displays different visual effects. Specifically, the prompts DeCap method discovered not only include the "sketch' category but also encompassed a wide range of domain information. Despite these imagos being visually distinct from real images, their classification performance significantly surpassed that of using only the "sketch' prompls, even approaching the performance achieved by training on the entire real dataset. This result demonstrates that diverse prompts help boost classification performance, and DeCap method genuincly focuses mining proper prompts for improving classification performance rather than visual effects. Even so, these selected prompts are hard to be interpreted by human vision. We hope that a rational theoretical insight could characterize such phenomenon in the future study.
D Visualization of Synthetic Images and Learned Prompts
D.1 Visualization of Synthetic Images
Fig. and Fig [ show more visualized examples of synthetic images Pets and Imagenet dalasets generated by DeCap method. As it shown, proposed DeCap method could generate diverse and multi- domain images, and thus obtain better classification performance.
D.2 More Visualization of Learned Prompts
We will demonstrate that DeCap method can adaptively learn proper and dataset-specific prompts that are suitable for concerned tasks from the following three aspects.
Firstly, the ratios of the number of model-generated and hand-crafted prompts for each class are varying, as shown Fig. and This reflects that our method could adaptively adjust the proportions that roconcile class/domain information and rich content information for different classes.
Secondly, though the hand-crafted prompts follow the same templates, may learn relatively different prompts in Fig.l. This further reveals our method could adaptively learn clasification-awaree prompts for each class, so as to achieve better performance on downstrcam tasks. More- over, we addilionally give some examples about the consistently selected model-generatedd prompls during the optimization process to further highlight the significance integrating fine-grained prompt descriptions. As we can see in Table [R. the consistently selected prompts show high diversity and fine-grained informa- tion, including: movement, posture, background, color, quantity, other objects, and so on. This pictures significantly help to provide classification-eenefit features.
that different classes
29

---

Published in Transactions on Machine Learning Research (09/2025)
Lastly, Table [+] shows that though STL-10 and CIFAR-10 datasets have some same categories, the learned prompts by our method could be almostly different. This demonstrated that our DeCap method could learning proper prompts suitable to concerned few-shot datasets. For instance, we can see that learned prompts for the STL-10 dataset are realistic, while Icarned prompts for CIFAR-10 datasct are of low- resolution imagery. Notice that these prompls are well aligned with prior knowledge of these datasets.
Moreover, we additionally display the complete set of prompt pool of the "airplane" class in STL-10 dataset in Table MT. to offer more intuitive understanding for the characteristic of our method stated above. And further give visualizations that demonstrate the prompt selection process over the course of optimization, including image examples and the evolution of prompts, please see Fig.IK
Figure 9: Examples of generated images on Pets dataset by DeCap method.
Figure 10: Examples of generated images on Imagenet 100 dataset by DeCap method.
30

---

Published in Transactions on Machine Learning Research (09/2025)
Comperison of Hend crafted and Model Gonereted Prompts
Hend-rrolted piompls
Model generated promats
11
14
14
1imbes
12
dog
Figure 11:
Illustrations of the number of hand-crafted prompts vs the number of model-genet generated prompts mined by DeCap method on STI.-10 dataset.
Comperison Hend crafted and Model Genereted Prompts
prompts
Model generated promets
13
13
12
12
13
11
14
Wimoerrs
10
0
Figure 12: Illustrations of the number of hand-crafted prompts vs the number of model-generated prompts mined by DeCap method on CIFAR10 dataset.
31

---

Published in Transactions on Machine Learning Research (09/2025)
Table 14: Illustration of mined prompts for "deer" class different datasets. DeCap method selects completely different prompts for the same class across different datasets, demonstrating its ability to adaptively learn the prompts suited to each specific dataset.
A deer is grazing the woods. deer are grazing under a tree a photo of the clean deer. silhouette of deer.
a deer and young man roam around during a december game a photo of deer.
deer in video game. toy deer a deer on a pond
the carloon deer.
STL-100
the hornets and deer are on a ridge a doer.
deer resting with the grazing padou atop old farmhouse An ink painting of deer deer on green pond. the toy deer. a brown bear eats the deer glossy deer.
a photo large deer.
a group of deer on prairie are seen grazing in their natural habitat
photo of deer, wild deer in the wild
deer farm soft-focus deer. art of a deer.
deer and their prey on the northern slopes a photo of deer, deer standing the snow with sky background pixelated photo of the deer. several deer grazing in the desert fox and deer on the grounds of city a rendering of a deer. a photo deer, a group of deers standing in a field
CIFAR10
silhouette of deer.
a photo of deer, a deer standing in the grass a deer is grazing an ancient inscription. photo of deer, herd of deer in the desert. deer and the munro.
A pair of deer on trail. a hunt deer on a desert land
door and the munro.
a pixelated photo of the deer.
32

---

Published in Transactions on Machine Learning Research (09/202))
5th iteration
10th iteration
20th iteration
40th iteration
60th iteration
80th iteration
(a) Image examples during different optimization iterations.
iteration selected prompts
10 itenniio selected procppts
20 iennsion sclaried prompis
prompts
60 iteniioo selectad peooppis
80 ienntion scloriell prompis
(b) The evalution prompts during different optimization iterations.
Figure 13: Tmage examples and the evolution of prompts during different optimization iterations. For clarify,
Fig.(b) (b) shows only the selected prompts and omits the rest.
33

---

Published in Transactions on Machine Learning Research (09/2025)
Table 15: Examples of model-generated prompts which consistently selected during optimization process.
cars that have to make black in room where the window down.
bird sitting branch.
effort turn
truck with lots people on it.
dog is standing in yard with harness on it.
A deer ls grazing the woods.
white horse in the open barn.
dogs inside a home on a
a semi truck driving down a rural road.
summer.
34

---

Published in Transactions on Machine Learning Research (09/2025)
Table 16: The prompt pool of "airplane" class in STL-10 dataset. Mined prompts by DeCap method are highlighted in bold.
Model-generated prompts
photo of airplane small plane is parked on the runway a photo of airplane a
photo of airplane large passenger jet fly- ing through a blue sky a photo of airplane, a small plane flying in the sky
photo of airplane yellow airplane flying small plane is on the runway
photo of airplane
through a blue sky a photo of airplane, a small plane is floating the water
a photo of airplane a
large passenger jet fly- ing through the sky
large passenger jet sit- ting on a runway
a photo of airplane small blue airplane
a photo of airplanc small plane flying in the sky
a photo of airplane a plane is parked on the tarmacl
photo of airplanc two small planes
laking off from the runway
sitting on the water
photo of airplane, a plane flying in the sky
photo of airplane, small plane flying over a mountain range a photo of airplane > two planes flying in the sky
photo of airplane a small plane flying through the air a photo of airplane, small plane flying over city
photo of airplane, plane is on the runway
a photo of airplane a large white plane
a photo of airplane a plane flying in the sky
a photo of airplane small plane flying
a photo of airplane a plane is parked on the tarmacl
a photo of airplane plane flying in the sky
photo of airplane
plane flying in the sky
through the air
photo of airplane small plane is parked on the water a photo of airplane a small plane flying in the sky
photo of airplane plane flying in the sky
photo of airplane plane flying in the sky
photo of airplane plane flying in the sky
a photo of airplanc small plane sitting on snowy field
An airplane that has been seen flying over another airplane.
A plane is in a parking lot.
small airplane is fly- ing over a highway time.
The plane has an en- gine, a seat, console, charger, and
sirplane that you
An airplane that is parked in an airport
bought a few years ago
The airplanes are all parked inside the park- ing lot.
An aircraft is in the flight over lake.
plane of small air- craft.
A plane is in the air.
A red and white air- plane with a green and green color scheme.
An airplane that has just broken ground be- hind it.
A plane parked next to one of the airplanes above it's engine.
An airplane parked the runway near pier.
A small airplane that's flying at low speeds under cloudy sky.
Airplanes in space that are not as big as usual.
An airplane parked along a highway.
airplanes need people to work hard at the zoo
The airplane is in green sky with blue skies.
The airplane with the lights is about to be docked.
An airplane parked next to bridge.
Some airplanes flying over people.
An airplane with three engines and pro- peller.
an airplane with window
An airplane parked on top of hill next to it
airplane on the tracks.
An airplane is parked on a runway at a air- port.
An airplane on an airplane track
plane with tires on it flying away from it.
These airplanes are in a wing.
a commercial airplane traveling in july.
airplane flight... photo and video
Two air planes all fly- ing in row.
modern airplane arriving in the air.
35

---

Published in Transactions on Machine Learning Research (09/2025)
An airplane in the mid- dle of nowhere with its doors lowered.
airplane is parked in a parking lot
An airplanc that
passengers in an air- plane in the rain
coming in to land.
a classic red blue air- plane is shown in the cockpit with bright col- as well.
airplanes flying at rale of 2 to 3 mph on sunday
An airplane on run- way next to small green field.
an airplane on an air- port runway
an airplane makes outgoing landing on
airplane and other ob- jects in the air
An old airplane is com- ing down the track.
planes in dry pit
the ground
A man attempting to board a commercial airplane.
Small airplanes with wing lights attached to them.
An airplane in a flight path with some pas- sengers nearby.
small airplane with the tail mounted up.
aircraft goes up through window dripping with smoke and debris.
airplanes that have been converted jet engines
airplane on an old building
airplanes cruising in the bay.
A boy is running with an airplane that is on the runway.
An airplane is about to land in parking lot and be delivered.
A blue airplane has its wings shut.
two airplanes parked at the airport
An airplane is sitting on a ground with all three engines on the ground.
There's one airplane in the cockpit which parked by another air- plane.
white airplane on the runway with blue ice.
jet airplane is ready for a test
aircraft carrier and an airplane together with some gulls.
An airplane in the air.
A family is on a small airplane at hotel.
airplane inside of the airplane
An airplane parked next to fireworks on the sky.
The airplane is looking down.
An airplane is shown flying on runway.
small bodied airplane on plane
An airplane that ap- pears to be on the run- way.
An airplane that is very close to the ground an airport.
Various aircraft and airplanes are getting ready for flight.
an airplane is seen ar- riving on a runway
Two aircrafts in
an airplane that making flying flight
airplane on the runway at the airport
white airplane station.
The airplane landed.
this airplane able to take off with just small amount of elfort to get the
An airplane that is in flying position.
An airplane making its way between jets.
airplane sitting in air
aircraft carrier and its crew arriving in an air- plane
a large old plane sits off the fuel tank
A family of airplanes are in a building.
airplane on the runway
an airplane that is making it's way around the tarmac jel airplane wing dur- ing maintenance
an airplane about to land in a desert
An electric airplane in the sky.
plane flies around city
airplanes the run- way
the of airplane board
The airplane is in the air.
A blue and white air- plane with white wing panels.
A commercial airplane flying under the radar.
The airplane has been damaged by the winds.
An airplane flying near tarmac.
Hand- craft-d prompts
36

---

Published in Transactions on Machine Learning Research (09/2025)
a good photo of the air- plane.
a photo of many air- plane.
a sculpture of a air- plane.
a photo of the hard to see airplanc.
a low resolution photo of the airplane.
a rendering of a air- plane.
a bad photo of the air- plane. a photo of a hard to see airplane.
graffiti of a airplane.
a cropped photo of the airplane.
the embroidered air- plane
tattoo of airplane.
a bright photo of air- plane.
photo of a clean air- plane.
photo of dirty air- plane
dark photo of the airplane.
photo of the cool air- plane.
drawing of airplane.
photo of my airplane.
the plastic airplane.
close-up photo of airplane.
black and white photo of the airplane. sculpture of the airplane.
painting of the air- plane.
painting of air- plane.
a pixclated photo of the airplane.
a bright photo of the airplane.
cropped photo of a airplane.
a photo of the dirty airplanc.
a jpeg corrupted photo of a airplanc.
a blurry photo of the airplanc.
a plastic airplane.
a bad photo a air- plane.
a rendering of the air- plane.
a airplane in a video game.
a photo of the airplane. a photo one air- plane.
a close-up photo of the airplane.
doodle of a airplane.
photo of airplane.
the airplane in
doodle of the air- plane.
the origami airplane. airplane.
sketch of airplane.
video game.
low resolution photo of airplane.
origami airplane.
the toy airplane.
a rendition of the air- plane.
photo of the clean airplane,
photo of large air- plane.
rendition of air- plane.
a photo of a nice air- plane.
photo of a weird air- plane.
a blurry photo of a air- plane.
a cartoon airplane.
a sketch of the air- plane.
a embroidered air- plane.
a pixelated photo of a airplane.
art of a airplane.
a jpeg corrupted photo of the airplane.
a good photo of air- plane.
a photo the nice air- plane.
a photo of the small airplane.
a photo of the weird airplane.
a drawing of the air- plane.
the cartoon airplanc.
of the airplane.
a photo of the large air- plane.
black and white photo of a airplane.
dark photo of air- plane.
graffiti of the airplane.
photo of cool air- plane.
photo of small air- plane.
tattoo of the air- plane.
toy airplane.
a modern style air- plane
an abstract photo of airplane
a digital style airplane
colorful airplane
a cartoon style air- plane
An ink painting of a airplane
a virtual style airplane
a toy airplane
A model airplane. a black airplane
a red airplane a white airplane
a blue airplane
a yellow airplane A futuristic airplanc.
An old airplane.
detailed illustration of airplane.
shadowy figure of airplane.
minimalist airplane.
close-up of airplane.
silhouette air- plane.
bright and vibrant airplane.
An abstract concept of airplane.
vintage style air- plane.
monochrome air- plane.
watercolor painting of airplane.
neon-lit airplane.
A sketch of airplane.
digital art of air- plane.
handerafted air-
An aerial view of air- plane.
A side profile of air- plane.
plane.
37

---

Published in Transactions on Machine Learning Research (09/2025)
A textured airplane.
A glossy airplane.
A
matte airplane.
A glowing airplane.
A rustic airplane.
weathered airplane.
sparkling airplane.
serene airplane.
A dynamic air- planc.
frozen moment of airplanc.
chaotic airplane.
whimsical airplane.
high-contrast air- plane.
soft-focus airplane.
sepia-toned airplane.
saturated airplane.
panoramic view of airplane.
An isolated airplanc.
mirrored airplane.
An enchanted airplane.
38


Reviews:
[{"review_id": "9DdbJ59cqI", "reviewer": "Reviewer_67KC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- (S1) The bilevel optimization problem being proposed is intuitive and interesting.\n- (S2) The Genetic Algorithm approach to solving the outer optimization loop is non-trivial.\n\nWeaknesses:\n\n- (W1) With regards to Equations 2 and 3, usually bilevel minimizations have an argmin and argmax so that the inner and outer objectives are adversarial. Here, both objectives minimize the same objective. Therefore it is unclear whether it is a meaningful objective. More detailed points:\n  - Are $\\L^{meta}$ and $\\L^{task}$ the same loss function (classification loss / cross entropy)? From my understanding they are. Please provide the exact formula for both if they are different. \n  - From my understanding, the solution to this optimization problem is simply the set of prompts that produce the training data exactly. For instance, I could tune the prompts such that the diffusion model outputs the training data (real data) exactly. Then, the optimization problem reduces to simple few-shot learning.\n\n- (W2) Table 1: How many shots? Please state in the table.\n\n- (W3) Table1: Your algorithm seems to require real training data. So, how does your algorithm handle the zero-shot scenario?\n\n- (W4) Figure 3: The Right most column example is not great. In my opinion, when the algorithm outputs prompts that explicitly mention datasets, it is likely overfitting to the task at hand. \n\n- (W5) I understand that the literature tends to focus on benchmarks like ImageNet, CIFAR, etc. However, I encourage the authors to think beyond the benchmarks presented here. The datasets used as benchmarks here are quite old and not very interesting.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors contribute a novel algorithm for ZS/FS multimodal classification. Specifically, the authors use a bi-level optimization approach to find a set of prompts, s.t. the synthetic images generated by a text-image-model using these prompts yield a strong classifier when used for training. They solve the outer loop of the optimization problem using genetic algorithms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please respond to W1.\n- Please add in details in response to W2 and W3.\n- I recommend finding better illustrative examples for Figure 3.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rkUK6kefXp", "reviewer": "Reviewer_gcxp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1. Tackles an important and timely problem: automating the generation of task-specific synthetic data.\n2. Leverages off-the-shelf diffusion models, making the approach more accessible and potentially generalizable.\n\n\n**Weaknesses:**\n\n1. In the introduction and methodology section, the paper argues that the proposed method is about learning to synthesize the prompts, however, I find the proposed method is merely a prompt sampling techniques from their pool.\n\n2. I am not fully convinced with the experiments designed in a few-shot learning framework to justify the importance of the proposed method. It is because few-shot  learning methods are useful when there is a possibility that manual annotation of a small number of samples is feasible. As we have the possibility to synthesize large amounts of data, the evaluation could be more impactful if conducted on large-scale classification tasks.\n\n3. Related work in GAN-based sampling and text-to-image generation is insufficiently discussed. Important references are missing, such as:\n      a. Bhattarai, Binod, et al. \"Sampling strategies for gan synthetic data.\" ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020.\n      b. Bodur, Rumeysa, et al. \"iedit: Localised text-guided image editing with weak supervision.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n      c. Bodur, Rumeysa, Binod Bhattarai, and Tae-Kyun Kim. \"Prompt augmentation for self-supervised text-guided image manipulation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method for automatically sampling task-specific text prompts to synthesize realistic images using pre-trained text-to-image diffusion models. The approach is based on a meta-learning framework that samples prompts suitable for downstream classification tasks Experimental results demonstrate competitive performance, particularly in few-shot learning settings.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify the methodology: the paper currently suggests a learning framework for prompt generation, but it appears to be more of a sampling-based approach. Terminology and framing should reflect this distinction accurately.\n\n2. Justify the choice of few-shot learning as a benchmark. Explain why synthetic data generation is meaningful in this context, or consider expanding the evaluation to include large-scale or more challenging tasks.\n\n3. Include and discuss relevant related work, particularly in the areas of prompt-based image generation and sampling synthetic data (e.g., Bhattarai et al., 2020; Bodur et al., 2024).", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4q3kj3vOSO", "reviewer": "Reviewer_uC6M", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Multi-level optimization seems to be a novel formulation to find the optimized set of prompts for few-shot task.\n- The diverse prompts would yield better quality images for learning down-stream tasks. \n\nWeaknesses:\n- Why not use some powerful image captioning models to generate various detailed level prompts and use these as prompts for generation. For instance, take a look at  multi-level image captioning part (fig 3) in https://arxiv.org/pdf/2409.10695 \n- In many instances, the CiP baseline is very competitive to the proposed method (see Table 1). It’s unclear if the complexity of the proposed scheme is necessary for these tasks ? \n- It is unclear why few-shot image classification requires diverse images?\n- Most likely with very detailed prompts, the diffusion model would start of hallucinate details. Its unclear how the proposed method handles such an issue.\n- CiP method is explored only with the BLIP2 captioning model, there are better captioning models used in current state-of-the-art diffusion models such as Molmo, InternLM, etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores the usage of off-the-shelf diffusion models to generate images for training non-specific downstream few-shot classification models to classify real images. Earlier works have relied on hand crafted prompts or model-generated text prompts to generate images. This typically yields images which are not diverse enough. This work explores a bi-level objective in which prompts are optimized to find a diverse set of images and inner level optimization is performed to learn down-stream task classifiers based on these synthetic images and few-shot task images. This proposed method is referred to as the Diversity-enhanced and Classification-aware prompt (DeCap) learning strategy to mine proper text prompts for downstream few-shot classification tasks. \n\nThis work uses a genetic search algorithm to solve the discrete prompt selection problem based on a pool of hand-crafted / model-generated prompts. Once the prompts are mined, these prompts are used to generate synthetic images and solve the few-shot tasks as inner optimization objective. This procedure (of find optimal prompts followed by image generation and few-shot task learning ) is repeated till convergence.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Questions: \n- Did the authors try other models than BLIP2 for CiP baseline?\n- How does the few-shot performance vary with the choice of diffusion model?\n- Do you have any comparison of the cost associated with the proposed method vs other baselines?\n- Why do few-shot image classification requires diverse images?\n- How does the proposed method handle hallucinations in images generated by diffusion model due to detailed prompts?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)