GeNIe: Generative Hard Negative Images Through Diffusion
Authors: Soroush Abbasi Koohpayegani
Anuj Singh
Navaneet K L
Hamed Pirsiavash
Hadi J. Rad
Venue: Accepted by TMLR ()

Abstract: Data augmentation is crucial in training deep models, preventing them from overfitting to limited data. Recent advances in generative AI, e.g., diffusion models, have enabled more sophisticated augmentation techniques that produce data resembling natural images. We introduce $\texttt{GeNIe}$ a novel augmentation method which leverages a latent diffusion model conditioned on a text prompt to combine two contrasting data points (an image from the source category and a text prompt from the target category) to generate challenging augmentations. To achieve this, we adjust the noise level (equivalently, number of diffusion iterations) to ensure the generated image retains low-level and background features from the source image while representing the target category, resulting in a hard negative sample for the source category. We further automate and enhance $\texttt{GeNIe}$ by adaptively adjusting the noise level selection on a per image basis (coined as $\texttt{GeNIe-Ada}$), leading to further performance improvements. Our extensive experiments, in both few-shot and long-tail distribution settings, demonstrate the effectiveness of our novel augmentation method and its superior performance over the prior art. Our code is available at https://github.com/UCDvision/GeNIe.

Published in Transactions on Machine Learning Research (04/2025)
GeNle: Generative Hard Negative Images Through Diffusion
Soroush Abbasi Koohpayegani" K L Navaneet!
Anuj Singh" 2 2
Hamed Pirsiavash
Hadi Jamali-Rad
University of California, Davis
Delff University of Technology, The Netherlands
Shhell Global Solutions International B. Amsterdam, The Netherlands
isoroush,nkadur hww.addrr..prrsssa//udddaasssss
f..r. h.a.r.siggh....jaallaaaadaad
Reviewed on OpenReview: https: http:://ppppprrreeee..e/////rrrr
Abstract
Data augmentation is crucial in training deep models, preventing them from overfitting to limited data. Recent advances in generative AI, e.g., diffusion models, have enabled more sophisticated augmentation techniques that produce data resembling natural images. We introduce GeNIe a novel augmentation method which leverages a latent diffusion model con- ditioned on a lext prompt lo combine two rasting data points (an image from the source category and a text prompt from the target category) to generate challenging augmenta- tions. To achieve this, we adjust the noise level (equivalently, number of diffusion itcrations) to ensure the generated image retains low-level and background features from the source im- age while representing the target category, resulting in a hard negolive sample for the source category. We further automate and enhance GeNIe by adaptively adjusting the noise level selection on a per image basis (coined as GeNIe-Ada), leading to further performance im- provements. Our extensive experiments, both few-shot and long-lail distribution settings, demonstrate the effectiveness of our novel augmentation method and its superior perfor- mance over the prior art Our code is available at: https:/ http:://gituubcom/ CoConn/CCDDvnnon/// DeeDiiison/GeNee GeNNe
1 Introduction
Data augmentation has become integral part of training deep learning models, particularly when faced with limited training data. For instance, when it comes to image classification with limited number of samples per class, model generalization ability can be siguificantly hindered. Simple transformations like rotation, cropping, and adjustments in brightness artificially diversify the training set, offering the model a more comprehensive grasp of potential data variations. Hence, augmentation can serve as a practical strategy to boost the model's learning capacity, minimizing the risk of overfitting and facilitating effective knowledge transfer from limited labelled data real-world scenarios. Various image augmentation methods, encompassing standard transformations, and learning-based based approaches have been proposed (Cubuk et al., 2019b;a; Yun et al., 2019b; Zhang et al., 2018; Trabucco et al., 2024). Some augmentation strategies combine two images possibly from two different categories to generate a new sample image. The simplest ones in this category are MixUp (Zhang et al., 2018) and CutMix (Yun et al., 2019a) where two images are combined in the pixel space. However, the resulting augmentations often do not lie within the manifold of natural images and act as out-of-distribution samples that will not be encountered during lesting.
Recently, leveraging generative models for data augmentation has gained upsurge of attention (Trabucco et al. 2024; Roy ct al., 2023; Luzi ct al., 2022; Ho et al., 2022b). These interesting studies, either based fine- tuning prompt engineering of diffusion models, are mostly focused on generating generic augmeniations
equal contribution

---

Published in Transactions on Machine Learning Research (04/2025)
Source image
Generated Images (Hard Negatives for the Source Image)
Training with Augmented Data
Torget Closs (dlg apple.... Fisk)
Promot: photo <Target Class?,
of
Aphaat arta mOUsE
Apher Ang
Classifier Model
Random Noise controlled by ratio r
Atooooo aroset
Aphita
A
Reverse Diffusion
Encoder
Oct esutee
Diffusion Model with Controlled Noise
inages
Samplias
Denkic
Figure 1: Generative Hard Negative Images Through Diffusion (GeNIe): generates hard negative images that belong the target category but similar to the source image from low-level feature and contextual perspectives. GelIee starts from a source image passing it through partial noise addition process, and conditioning it different targel calegory. By controlling the amount of noise, the reverse latent diffuusio process generates images that serve as hard negatives for the source category.
without considering the impact of other classes and incorporating that information into the generative process for a classification context. We take a different approach to generale challenging augmentations near the decision boundaries of downstream classifier. Inspired by diffusion-based image editing methods (Meng et al., 2021; Luzi et al., 2022) some of which are previously used for data augmentation, conditional latent diffusion models (Rombach et al., 2021a) for generating hard negatice images. Our core idea (coined as GeNIe) is to sample source images from various categories and prompt the diffusion model with a contradictory text corresponding to different target category. We demonstrate that the choice of noise level (or equivalently number of iterations) for the diffusion process plays a pivotal role in generating images that semantically belong to the larget category while retaining low-level features from the source image. We argue that these generated samples serve as hard negatives (Xuan et al., 2021; Mao al., 2017) for the source category (or from a dual perspective hard positives for the target category).. To further enhance GeNIe, propose an adaptive noise level selection strategy (dubbed GeNIe-Ada) enabling it to adjust noise levels automatically per sample.
propose to
To establish the impact of GeNIe, we focus on two challenging scenarios: long-tail and few-shol settings. In real-world applications, data often follows long-tail distribution, where common scenarios dominate and rare occurrences are underrepresented. For instance, a person jaywalking a highway causes models to struggle with such unusual scenarios. Combating such bias lack of sufficient data samples during model training is essential in building robust models for applications such as self-driving cars. Same challenge arises in few-shot learning settings where the model has to learn from only handful of samples. Our extensive quantitative and qualitative experimentation, on suite of few-shot and long-tai tail distribution scttings, corroborate the evfectiveness of the proposed novel augmentation method (GeNIe, GeNIe-Ada) in generaling hard negatives, corroborating its significant impact on categories with a limited number of samples. A high-level sketch of GeNIe illustrated in Fig. Our main contributions are summarized below:
We introduce GeNIe, a novel yet elegantly simple diffusion- based augmentation method to create challenging hard-negative augmentations in the manifold of natural images. For the first time, to best knowledge, GeNIe achieves this by combining two sources of information (a source image, and contradictory target prompt) through noise-level adjustment mechanism in the diffusion-denoisingg process.
We further extend GeNIe by automating the noise-level adjustment strategy on a per-sample basis (called GeNIe-Ada), enable generating hard negative samples in the context of image classification, leading also to further performance enhancement.
2

---

Published in Transactions on Machine Learning Research (04/2025)
To substantiate the impact of GeNIe, we present suite of quantitative and qualitative results including ex- tensive experinentation ion two challenging tasks: few-shot and long tail distribution settings corroborating that GeNIe (and its extension GeNIe-Ada Addle--Aaa significantly improve the downstream classification performance.
2
Related Work
Data Augmentations. Simple flipping, cropping, colour jittering, and blurring are some forms of image augmentations (Shorten & Khoshgoftaar, 2019). These augmentations are commonly adopted in training deep learning models. However, using these data augmentations is not trivial in some domains. For ex- ample, using blurring might remove important low-level information from medical images. More advanced approaches, such as MixUp (Zhang et al., 2018) and CutMix (Yun et al., 2019a), mix images and their labels accordingly (Hendrycks et al., 2020; Liu et al., 2022; Kim et al., 2020; Cubuk et al., 2020). However, the resulting augmentations are not natural images anymore, and thus, act as out-of-distribution samples that will not be seen at test time. To combat this, SAGE (Ma et al., 2022) proposes a data augmentation technique that uses visual saliency to perform optimal image blending at each spatial location, and optimizes the relative image position such that the resulting visual saliency is maximized. Another strand of research tailors the augmentation strategy through learning process to fit the training data (Ding et al., 2024; Cubuk et al, 2019b;a). Unlike the above methods, we propose to utilize pre-trained latent diffusion mod- cls to generate hard negatives (in contrast to generic augmentations) through a noise adaptation strategy discussed in Section 3.
Data Augmentation with Generative Models. Using syuthesized images from generative models to augment training data has been studied before in many domains (Frid-Adar et al., 2018; Sankaranarayanan et al., 2018), including domain adaptation (Huang et al., 2018), visual alignment (Peebles et al., 2022), and mitigation of dataset bias (Sharmanska et al, 2020; Hemmat et al., 2023; Prabhu et al., 2024). For example, (Prabhu et al., 2024) introduces a methodology aimed at enhancing test set evaluation through augmentation. While previous methods predominantly relied on GANs (Zhang et al., 2021c; Li et al., 2022b; Tritrong et al., 2021) as the generative model, more recent studies promote using diffusion models to augment the data (Rombach et al., 2021a; He et al., 2022b; Shipard et al., 2023; Trabucco et al., 2024; Azizi et al., 2023; Luo et al., 2023; Roy et al., 2023; Jain et al., 2022; Feng et al., 2023; Dunlap et al., 2023b; Chegini & Feizi, 2023). More specifically, (Trabucco et al., 2024; Roy et al., 2023; He et al., 2022b; Azizi al., 2023) study the effectiveness of text-to-image diffusion models in data, augmentation by diversification of each class with syuthetic images. (Roy et al., 2023) also utilizes a text-to-image diffusion model, but with BLIP (Li et al., 2022d) model to generate meaningful captions from the existing images. (Jain et al., 2022) utilizes diffusion models for augmentation to correct model mistakes. (Feng et al., 2023) uses CLIP (Radford et al., 2021) to filter generated images. Generative models for data augmentation may produce dut-of-distributiin samples if the rewnstream task's data distribution differs. Fine-tuning on a small rewnsstream dalaset can address this. For example, DAFusion (Trabucco et al., 2024) fine-tunes diffusion model using textual inversion (Gal et al., 2022a), while SiSTA (Thopalli et al., 2023) adapts a GAN for the task. (Graikos et al,, 2023a) propose adapting generative models to downstream tasks by leveraging the internal representations of the denoiser network. Investigations by (Tian et al., 2023) explore the use of text-to-image synthetic images for generating positive samples in contrastive learning. (Dunlap et al., 2023b) utilizes text-based diffusion and a large language model (LLM) to diversify the training data. (Chegini & Feizi, 2023) uses an LLM to generate text descriptions of failure modes associated with spurious correlations, which are then used to generate synthetic data through generative models. The challenge here is that the LLM has little understanding of such failure scenarios and contexts.
We take completely different approach here, without replying any extra source of information (e.g., through an LLM). Inspired by image editing approaches such as Boomerang (Luzi et al., 2022) and SDEdit (Meng et al., 2021), we propose to adaptively guide a latent diffusion model to generate hard negatives images (Mao et al., 2017; Xuan et al., 2021) on a per-sample basis per category. In a nutshell, the aforementioned studies focus on improving the diversity of cach class with effective prompts and diffusion models, however, we focus on generating effective hard negative samples for each class by combining two sources of contradicting information (images from the source category and text prompt from the target category).
3

---

Published in Transactions on Machine Learning Research (04/2025)
Language Guided Recognition Models. Vision- Language foundation models (VLMs) (Alayrac et al., 2022; Radford et al., 2021; Rombach et al., 2021a; Saharia et al., 2022; Ramesh et al., 2022; 2021) utilize human language to guide the generation of images or to extract features from images that are aligned with human language. CLIP (Radford et al., 2021) excels in zcro-shot tasks by aligning images with text, while recent works improve prompts (Dunlap el al., 2023a; Petryk et al., 2022) or use diffusion models as classifiers (Li et al., 2023). Similarly, we leverage Stable Diffusion 1.5 (Rombach et al., 2021a) to enhance downstream tasks by augmenting training data with hard negative samples based on category names.
Few-Shot Learning. In Few-shot Learning (FSL), we pre-train a model with abundant data to learn a rich reprosentation, then fine-tune it on new tasks with only a few available samples. In supervised FSL (Chen et al., 2019a; Afrasiyabi et al., 2019; Qiao et al., 2018; Ye et al., 2020; Dvornik et al., 2019; Li et al., 2020; Sung et al., 2018; Zhou et al., 2021; Singh & Samali-Rad, 2023), pretraining is done on a labeled dataset, whereas in unsupervised FSL (Jang et al,, 2022; Wang & Deng, 2022; Lu et al., 2022; Qin et al., 2020; Antonion & Storkey, 2019; Khodadadch et al., 2019; Hsu et al., 2018; Medina et al., 2020; Shirekar et al., 2023) the pretraining has to be conducted on an unlabeled datasel. posing an extra challenge in the learning paradigm and neighboring these methods closer to the realm of self-supervised learning.
3 Proposed Method: GeNIe
Given a source image Xs from category S = <source category>, we are interested in generating a target image X, from category T = <target category>. In doing so, we intend to ensure the low-level visual features or background context of the source image are preserved, so that we generate samples that would serve as hard negatives for the source image. To this aim, we adopt a conditional latent diffusion model (such as Stable Diffusion, (Rombach ct al., 2021a)) conditioned on text prompt of the following format "A photo of T = <target category>".
Key Idea. GeNIe in its basic form is simple yet effective augmentation sample generator for improving a classifier fo(.) with the following two key aspects: (i) inspired by (Luzi et al., 2022; Meng et al., 2021) instead of adding the full amount of noise max and going through all Nmaz (being typically 50) steps of denoising, we use less amount of noise (romar, with r E 10) and consequently fewer number of denoising iterations [[r:man]];; (ii) we prompt the diffusion model with a P mandating a target category T different than the source S. Hence, we denote the conditional diffusion process as X, (XDiff(Xs,P.r) In such construct, the proximity of the final decoded image X, to the source image Xs the target category defined through the text prompt P depends on r. Hence, by controlling the amount of noise, we can generate images that blend characteristics of both the lext prompt. P and the source image Xs. If we do not provide much of visual details in the text prompt (e.g., desired background, etc.), we expect the decoded image X, to follow the details of Xs while reflecting the somantics of the text prompt P. We arguc, and demonstrato later, that the newly generated samples can serve as hard negalive examples for the source category S since they share the low-level features of Xs while representing the semantics of the target category, T. Notably, the source category S can be randomly sampled or be carefully extracted from the confusion matrix of fe(.) based on real training data, The latter might result in even harder negative samples being now cognizant of model confusions. Finally, we will append our initial dataset with the newly generated hard negative samples through GeNIe and (re)train the classifier model.
Enhancing GeNIe: GeNIe-Ada. One of the remarkable aspects of GeNIe lies in its simple application, requiring only Xs: P, and r. However, selecting the appropriate value for r poses a challenge as it profoundly influences the outcome. When is small, the resulting X, tends to closcly resemble Xs, and conversely, when is large (closer to 1), it tends to resemble the semantics of the larget category. This phenomenon arises because smaller noise level restricts the capacity of the diffusion model to deviate from the semantics of the input Xs. Thus, critical question emerges: how can we select for a particular source image to generate samples that preserve the low-level semantics of the source category S in Xs while effectively representing the semantics of the larget category T? We propose a method to determine an ideal value for r.
Our intuition suggests that by varying the noise ratio from 0 to 1, X, will progressively resemble category S in the beginning and category T towards the end. However, somewhere between 0 and 1, X, will undergo rapid transition from category S to T. This phenomenon is empirically observed in our experiments with
4

---

Published in Transactions on Machine Learning Research (04/2025)
Prompt: photo motorcycle.
Source Image
GeNIe (40.4)
GeNIe (+08))
GeNIe (n:0.))
(nip.))
GeNIe
Prompt: A photo a GeWIe (--0.)
Source Image
GeNIe (moo4)
GeNIe (md
GeNIe (nip.8)
GeNle (=-0.))
Figure 2: Effect of noise ratio, in GeNIe: employ GeNIe to generate augmentations for the target classes (motorcycle and cat) with varying Smaller r yields images closely resembling the source semantics, creating inconsistency with the intended target label. By tracing from 0 to 1, augmentations gradually transition from source image characteristies to the larget category. However, a distinct shift from the source to the target occurs at a specific r that may vary for different source images or target categories. For more examples, please refer to Fig. A9,
A Algorithm 1: AoNIe-Ada
Require: Xs. Xy. fol.), STDLEf(), M
Zowh??
Extract 75 +- (o( Zr + Jo(Nr)
Ze
for m E [I do
N: Z, - (x(STDLE((XXP,1)
dm - (2, 12,,22)"12,-22,
2s1la
argmax.. Im dm the Vin M2
Return: X,- P.D.
Z
Figure 3: GeNIe-Ada: To choose adaptively for each (source image, target category) pair, propose tracing the semantic trajectory from 7s (source image embeddings) to Zr (target embeddings) through backbone feature extractor fo(-) (Algorithm 1). We adaptively select the sample right after the largest semantic shift.
varying т, as depicted in Fig. Although the exact reason for this rapid change remains uncertain, one possible explanation is that the intermodiate points between two categorics reside far from the natural image manifold, thus, challenging the diffusion model's capability generate them Selocchi et al. (2024). Ideally, should select corresponding to just. after this rapid semantic transition, at this point, X, exhibits the highest similarity to the source image while belonging to the target category.
We propose to trace the semantic trajectory between Xs and XT through backbone feature extractor fo(.). As shown in Algorithm 1, assuming access to the classifier backbone fo(.) and at least one example Xy from the target category, we convert both Xs and Xr into their respective latent vectors Zş and Zr by passing them through fo).). Then, we sample M values for r uniformly distributed (0,1), generating their corre- caleulate d, (2.-2s)* (2, 1|2r-2sll
sponding X, and their latent vectors Z, for all those Subsequently, as the distance between Z, and Zs projected onto the vector connecting Zg and Zr. Our hypothesis posits that the rapid semantic transition corresponds sample values for uniformly distributed between 0 and 1, and analyze the variations in d,. We identify the largest gap in d, and select the r value just after the gap when increasing r, detailed in Algorithm and illustrated in Fig. 3.
sharp change in this projected distance. Therefore, we
3.1 Psuedocode of GeNle:
As illustrated Algorithm 2, we provide a detailed pytorch-style pseudocode for GeNIe. First, a SDv1.5 pipeline initialized by loading all the components such the VQ-VAF encoder and decoder, the CL.P text.
5

---

Published in Transactions on Machine Learning Research (04/2025)
encoder and the DPM scheduler for the forward and reverse diffusion process. Then, the source image is input to the encoder to encode the image into latent space for the diffusion model. Next, the encoded image is partially noised based on the noise ratio r using the scheduler. The diffusion model then de-noises the partially noised latent embedding for a total of NUM INFERENCE STEPS xr steps, with an additional input of a lext prompt from a contradictory larget class. Finally, the decoder decodes the de-noised latent embedding into the generated hard-negative image, that contains the low-level features of the source image and the class/categos categoryy of the contradictory text-prompt.
Algorithm 2: Py PyTorch-style Pseudocode of GeNle.
StableDiffus oon wwtssiionppiipiiie ipeline: Pre-trained diffusion nodel wtw.oreeeeeetsseeeshieee
Scheduler for forvard and reverse diffusion ancade, latents: Encodes an image into latent space
decode secode_latenta latents: Decodes latenta back into inage
def AugmentGellI (saurce, image, target_proapt. proupt, parcent parcent_noisa): moise): 50 Number of steps for reverse diffusion
XUN INPEREKCE_ NUM TRAIN_ STEPS 1000 Number of steps
forvard diffusion
# Initialize the stable diffusion pipeline and scheduler pipe PiablebiffasoonPpplll wwt.tppaaaaa..... fron pratrained "sable-diifusstoo-1166"" acheduler htt.:rerrrrtttttpescceeeeeeeee www.froooooooioiioiooooooo. (pppi scheduler.com
# Encode the latenta encode_ (etdd_ltttnttteure__ooaaoo
image into latent space
# Forvard Diffusion
noise Ferch.rannnnlasee shape) # Generate random noise
tinestep = torch Tensor [int (NUM TRAIN STEPS percent, [oise)]] a Calculate timeatep latents_nokt _ooitts__eiee__ acheduler. add (1atenta,,,a, noise, tineatep) Add noise latenta
# Reverse Diffusion
latenta pipe<
pronpt-target_proup
percent_noiie soseetoooooorccttnnniiii.
latants=latenta, noise,
nun SeteerenneesstssNleee
INFERENCE_STER
Decode
augnented Inage
augnented_inee image
lateetsstttttt
return augnented_ inage
4 Experiments
Since the impact of augmentation is more pronounced when the training data is limited, we evaluate the impact of GeNIe on Few-Shot classification in Section 4.1, Long-Tailed classification in Scction 4.3, and Section 4.2. For GeNIe-Ada in all scenarios, we utilize GeNIe generate aug-
fine-grained classification mentations from the noise level set {0.5, 0.6, 0.7, 0.8. 0.9). The selection of the appropriate noise level per source image and target is adaptive, achieved through Algorithm 1.
Baselines. We use Stable Diffusion 1.5 (Rombach et al., 2021a) as our base diffusion model. In all settings, we use the same prompt format to generate images for the target class: iee. "A photo of a <target category>", where we replace the target category with the target category label. We gener- ate 512 x 512 images for all methods. For fairness, we generate the same number of new images for each class. We use a single NVIDIA RTX 3090 for image generation. We consider 4 diffusion-based baselines and suite of traditional data augmentation baselines.
Img2Img (Luzi et al., 2022; Meng et al,, 2021): We sample an image from target class, add noise to its latent representation and then pass it along with prompt for the target category through reverse diffusion. The focus here is on a target class for which we generate extra positive samples. Adding large amount of noise leads to generating an image less similar to the original image. We use two different noise magnitudes for this baseline: r = 0.3 and 0.7 and denote them by Img2Img' and Img2Img", respectively.
Txt2Img (Azizi al., 2023; He al., 2022b): For this bascline, we omit the forward diffusion process and only use the reverse process starting from text prompt for the target class of interest. This is similar to the

---

Published in Transactions on Machine Learning Research (04/2025)
base text-to-image generation strategy adopted in (Rombach et al., 2021a; He et al., 2022b; Shipard et al., 2023; Azizi et al., 2023; Luo et al., 2023). Fig. 4 illustrates a set of generated augmentation examples for Txt2Img, Img2Img, and GeNIe.
DAFusion (Trabucco et al., 2024): In this method, an embedding is optimized with a set of images for each class to correspond to the classes the dataset. This approach introduced in Textual Inversion (Gal et al., 2022c). We optimize an embedding for 5000 iterations for each class in the dataset, followed by augmentation similar as the DAFusion method.
Cap2Aug(Roy et al., 2023): It is recent diffusion-based data augmentation strategy that uses image captions as text prompts for an image-to-inen io-iage diffusion model.
Traditional Data Augmentation: We consider both weak and strong traditional augmentations. More specifically, for weak augmentation we use random resize crop with scaling E [0.2, 1.0] and horizontal flipping. For strong augmentation, we consider random color jitter, random grayscale, and Gaussian blur. For the sake of completeness, we also compare against data augmentations such as CutMix (Yun el al,, 2019a) and MixUp (Zhang et al., 2018) that naively combine two images together, and SAGE (Ma et al., 2022), that mixes image pairs based on their visual saliency to promote discriminative foreground objects in the mix.
4.1 Few-shot Classification
We assess the impact of GeNIe compared to other augmentations in a number of few-shot classification (FSI.) scenarios, where the model has to learn only from the samples contained in the (N-way, K-shot) support set and infer on the query set. Note that this implies that the GeNle, GeNle-Ade augmentations are only applied during few-shot inference on the test-datasct, and not during the pre-training stage, where the backbone is unsupervised pre-trained on an abundant (lew-shot training) dataset. The goal is assess how well the model can benefit from the augmentations while keeping the original N K samples intact.
We conduct our few-shot experiments on two most commonly adopted few-shot classification datasets: mins--magenet, (Ravi & Larochelle, 2017) and liered-Imagenet (Ren et al., 2018). mini-Imagenet is a subset of ImageNet (Deng et al., 2009) for few-shot classification. It contains 100 classes with 600 samples each. We follow the predominantly adopted settings of (Ravi & Larochelle, 2017; Chen et al., 2019a) where we split the entire dataset into 64 classes for training, 16 for validation and 20 for testing. tiered- Imagenet is a larger subset of ImageNet with 608 classes and total of 779, 165 images, which are grouped into 34 higher-level nodes in the ImageNet human-curated hicrarchy. This set of nodes is partitioned into 20, 6, and 8 disjoint sets of training, validation, and testing nodes, and the corresponding classes form the respective meta-sets.
Datasets,
Evaluation. We evaluate the test-sct accuracies of a statc-of-tho-art unsupervised few-shot learning method with GeNIe and compare them against the accuracies obtained using other augmentation methods. Specif- ically, we use UniSiam (Lu et al., 2022) pre-trained with ResNet-18, ResNet-34 and ResNet-50 backbones and follow its evaluation strategy of fine-tuning logistic regressor to perform (N-way, K-shot) classification the test sets of miné- and diered-Imagenn Following (Ravi & Larochelle, 2017), an episode consists of a labeled support-set and an unlabelled query-set. The support-set contains N randomly sampled classes where each class contains K samples, whereas the query-set contains Q randomly sampled unlabeled images per class. We conduct our experiments on the two most commonly adopted settings: (5-way, 1-shot) and (5-way, 5-shot) classification settings. Following the literature, we sample 16-shots per class for the query set in both settings. We report the test accuracies along with the 95% confidence interval over 600 and 1000 episodes for méni-ImageeNet and lèered-ImageeNet, respectively.
Implementation Details: GeNIe generates augmented images for each class using images from all other classes as the source image. We use r 0.8 in our experiments. We generate 4 samples per class as augmentations in the 5-way, 1-shot setting and 20 samples per class as augmentations ions in the 5-way, 5- shot setting. For the sake of a fair comparison, we ensure that the total number of labelled samples in the support set after augmentation remains the same across all different traditional and generative augmentation methodologics. Due to the expensive training of embeddings for each class in each episode, we only evaluated the DA- DA-Fusion baseline on the first 100 episodes.
7

---

Published in Transactions on Machine Learning Research (04/2025)
1mg2 Amg
1mg2 imgff
Txt 2 Img
Source Image mushroom
GeNIe (m0.66
GeNIe (m0077 (m-0.8)
volcano
goldfish
snoek
perman shepherd
siberian husky
chocolate sauze
New C
red wine
powder
Figure 4: Visualization of Generative Samples: We compare GelIee with two baselines: Img2Img" augmenta-
tion: both image and text prompt are from the same category. Adding noise does not change the image much, so they are not hard examples. Txt2Ing augmentation: We simply use the text prompt only to generate an image for the desired category (e.g., using a text2image method). Such images may be far from the domain of our task since the generation not informed by any visual data from our lask. GeNIe augmentation: We use the larget category name in the text prompt only along with the image.
Results:
The results on mini-Imagenet and tiered-Imagenet for both (5-way, and 5-shot) settings are summarized in Table 1 and Table 3, respectively. Regardless of the choice of backbone, we observe that GeNIe helps consistently improve UniSiam's performance and outperform other supervised and unsupervised few-shot classification methods as well as other diffusion-basse (Trabucco et al., 2024; Luzi et al., 2022; Rombach et al., 2021b; He et al,, 2022b) and classical (Yun et al., 2019a; Zhang et al., 2018; Ma et al., 2022) data augmentation techniques on both datasets, across both (5-way, and 5-shot) settings. Our noise adaptive method of selecting optimal augmentations per source image (GeNIe-Adal further improves GeNIe's performance across all three backbones, both few-shot settings, and both datasets (mini and liered- Imagenet).
4.2 Fine-grained Few-shot Classification
To further investigate the impact of the proposed method, we compare GeNIe with other text-based data augmentation techniques across four distinct fine- fnnegraiind datasets in 20-way, 1-shot classification setting, We employ the pre-trained DINOV2 VIT-G (Oquab et al., 2023) backbone feature extractor to derive
8

---

Published in Transactions on Machine Learning Research (04/2025)
Table I: mini- ImageNNt: We use our augmentations on (5-way, 1-shot) and (5-way, 5-shot) few-shot settings of mini- Imagenet dalaset with 3 different backbones (ResNel-18, 34, and 50). We compare with various baselines and show that our augmentations with UniSiam outperform all the baselines including Txt2Ing and DAFusion augmentation. The number of generated images per class is 4 for 1-shot and 20 for 5-shot settings.
ResNet-18
BexNoi--44
Augisentation
Method
Pre-troining
1-sbot
5-shot
Augmentation
Method
Pre-training
1-shot
G-shot
Weak
Baseline 2010s.
sup.
40.5:107
70.326.7
iDeMe-Net 2019b
sup.
50.1=0.9
74.6+0.7
Weak
Baslliar++ 2010
52.7:0.5
70.246.6
Robust + dist 2019
sup.
63.720.6
81.210.4
Weak
SmaCR 2000
ussup
64000044
70.860.3
AFIIN 2020
sup.
62.4=0.7
78 78.240.6
Weak
SimSiars 2n2]
655:40.4
50,146.5
Weak
ProtoNec+SSL 2020
sup-tesl
76.6
Weak
UniSom-dies 2022
05.6:40.4
K3.4:022
Wesk
Neg. CreeCCeine 2020
0233:08
80.920.6
Weak
UniSoom 5022
unsup
64.3:105
523360.5
Controid Alig22119
sup. 1..9-0.7
80.4+0.7
Strong
UniSoom 3022
64.2:10.5
Weak
PSST 2021
Buseline---4 2018
Buseline 2019h
sup. [se
sup.
100-0.8
50.5=0.5
50.6:0.8
77.4+0.5
76.7+0.6
17.31:066
Irg2Isg
Cutbiis 2015a
MixUpp 2018
lngging
UniSaa 3022
UniSam 2022
UniSiom 3022
UuiSam 2022
uneug
unmp
uuzu.
64.000.5
657710.5
65.540055
70.5:0.5
51.746.6
54.56.55
50.110.8
$2.940.5
Wook
Wook
Werk
SimCLR 2020
ProtoCLR 20720
UMTRA 2019
unup.
maup.
meep-
50.9-0.4
62.6=0.4
40.1=0.4
71.6+0.3
50.4+0.3
79:7400.3
MeMIe-Mda (Ouns)
Sella (Ours)
DAPuska 2024
Teeategliins 20156
UniStam 502
UniSaam 3022
UniSam 2022
UniSao 2022
unmp
unep
unsup
uasup
78.5:10.6
77.1-2008
75.440.55
64.7111.9
86.6:004
66.5:0.41
55.540.5
50.2411.4
Weak
SimSSiam 2021
mamp.
62.8=0.4
79.9.10.3
RewNee--0
Weak
UuiSiamn+dist 2022
nasup.
61.110.4
82.310.33
Wook
PDA-Ne 7021
mnnpg
65840.9
35.11..6
Weak
UniSiisn 2022
naeup.
63. 1::088
81.1::0.55
Weak
Meta. DM 2023
unsup
66.7:40.4
55.330.22
Strong
UniSiam 2022
uasup.
62.8.
81.210.6
Wook
UniSaam 3022
unerp
63:-10.5
S3,110.5
MuiMix 2019s
UniSisan 2022
masup
02,7=0.8
80.6:006
Stoong
UniSamm 3022
unsup.
64.8:10.5
S1.230.5
Mix p 2018
UniSiam 2022
uneup
62:1-0.8
80.7+0.6
GatMia 2010s
UniSaa 2022
unsup
64:3:005
53.236.5
TrggTTng" 20222
Ing? Ing 2012
UniSisan 2022
UniSiae 2022
uneup
mesp.
68.1-0.7
63.9:0.88
84.0+0.3
82.1:005
SXGE 2002
MhdUp 2018
Ing2ing 2002
UniSiam 2022
UniSiam 2022
UniSlam 5522
unsup
unmp
75.720..
63550055
60.0:005
54.920.4
51,000.5
536-00.5
2020; 2022b
UniSlam 2022
mnsp-
74.1=0.6
84.6++0.5
Iegatig 2uzz
UniSoam 3042
71:1:10.7
55.740.5
DAFusion 2024
UniSiam 2022
unsup
64.3=18
82.01114
2es2 20220
UniSam 2022
76.4:0.6
50.526.4
Gallaa (Ours)
UniSiam 2022
unsp-
75.5+0.66
86.4+00.4
DAPudkn
UniSam 3022
uneu
65.7+1.5
53.9+1.2
GeMle-Ada (Ours)
UniSiam 2022
unsup.
76.810.6
85.910.4
SoMZe-Mda (Ours)
BoMZo (Owzs)
UniSSon 2022
UniStam 2022
unmp
unsap
77.3:00.6
78.6:10.6
2:124
87.9:00.4
features from training images. Subsequently, an SVM classifier is trained on these features, and we report the Top-1 accuracy of the model on the test set.
Table 2 summarizes the results. Addi- tional details about this experiment can be found in Section A.9. GeNIe outperforms all other base- lines, including Txt2Img, by margins upto 0.5% on CUB200, 6.6% on Cars196, 0.1% on Food101 and 5.3% on FGvC-Aircraft GeNIe exhibits great ef- foctiveness in more challenging datasots, outper- forming the baseline with traditional augmentation by about 38% for the Cars dataset and by roughly 17% for the Aircraft dataset. It can be observed here that GeNIe-Ada performs un-par with GeNIe with a fixed noise level, eliminating the necessity for noise level search in GeNIe.
Results:
Table 2: Few-shot Learning on Fine-grained dataset: We utilize an SVM classifier trained atop the DINOV2 ViT- G pretrained backbone, reporting Top-1 accuracy for the test set of each dataset. The baseline is an SVM trained on the same backbone using weak angmentation.
Method
Birds
Cars
Foods
Aireraft Alreraft 2013
CUB200 2011
Cams196 2013
Food101 2014
Baseline
90.3
49.8
82.9
29.2
20022
90.7
50.4
87.4
31.0
"2022
91.3
56.4
91.7
34.7
10222ng20226
92.0
81.3
93.0
41.7
GeliIe (1=0.5)
81.6
91.3
GaMlee (r-0.6)
92.2
87.1
92.5
45.0
GeliIe (r=0.7)
92.5
87.9
92,9
47.0
Galia (r-0.8)
92.5
46.5
GelIee (r=0.9)
92.4
87.1
93.1
45.7
GellIe Ada
92.6
87.9
93.1
16.9
4.3 Long- Tailed Classification
We evaluate our method on long-tailed data, where the number of instances per class is unbalanced, with most categories having limited samples (tail). Our goal is to mitigate this bias by augmenting the tail of the distribution with generated samples. We evaluate GeNIe using two backbones: ViT with LViT (Xu et al., 2023) and ResNet50 with VL-LTR (Tian et al., 2022). Following LVTT, we first train an MAE (He et al., 2021) and VIT on the unbalanced dataset without any augmentation. Next, we train the Balanced Fine- Tuning stage of LVTT by incorporating the augmentation data generated using GeNIe or other baselines. For ResNet50, we use VL-LTRR code to fine-tune the CLIP ResNet50 with generated augmentations by GeNIe.
Dataset: We perform experiments on ImageNet-LT (Liu et al., 2019). It contains 115.8K images from 1, 1,000 categories. The number of images per class varies from 1280 to 5. Imagenet-LT classes can be divided into groups: "Few" with less than 20 images, "Med" with 20 100 images, and "Many" with more than 100 images. Imagenet-LT uses the same validation set as ImageNet. We augment "Few" categories only and limit the number of generated images to 50 samples per class. For GeNIe, instead of randomly sampling
9

---

Published in Transactions on Machine Learning Research (04/2025)
Accuracies (% + std) for 5- way, 1-shol and 5-way, 5-shot classification setlings on the test-set. We compare against various SOTA supervised and unsupervised few-shot classification baselines as well as other augmentation methods, with UniSiam (Lu et al,, 2022) pre-trained ResNet- 1eee--18500 18,50 backbones.
Table 3: tiered-Imagee
ImagoNet-LT: We com- pare different augmentation methods on ImageNet- LT and report Top-1 accuracy for "Few", "Medium", and "Many" sets. On the "Few" set and LiVT method, our augmentations improve the accuracy by 11.7 points compared to LiVT original augunentation and 4.4 points compared to Txt2Ing. GeNIe- out- performs Cap2Ang baseline in "Few" categories by 7.6%. Refer to Table All for full comparison with prior Long-Tailed methods.
Table 4:
ResNet-11
Angmentation
Method
Pre-training
L-shot
S-ahot
Weak
SinCLR82000
63.4214
79.250.3
Wook
SiuSism 2021
moorg..
$1.6+0.3
Woak
UniSlam 2122
utsup.
ResNet-50
Strong
CutMie 2010s
UniSam 2022
UniStam 2022
mmmp.
mndu..
02:120.7
62810.7
78.920.6
80.910.5
Method
Many
Med.
Few
Overall Ace
MieeUp 2018
UniSiam 2022
02.1+007
78.4+0.8
ResLT 2022
63.3
53.3
40.3
55.1
Teg2Tag f2022
Ieg2Ing
Farooo 2024
12:2Ing.1122
UniSlam 2122
UniStam 2122
UniStam 2022
UniSlam 2022
mmmp.
mesuu..
uosup.
08.7211.7
03.+00.7
72.5115
6266121
84.220.5
51.8-0.5
$1.0115
53.52-0.5
Zemo-shot CLI 2011
LWS 2019
PaCo 2021b
62.2
60.8
68.2
38.7
30.3
48.6
31.8
08.6
60.0
51.5
GeltIe(Ours)
UniStam 2022
0..610.6
65.0.10.4
DRO-ET 2021
64.0
49.8
33.1
$1.5
Ouno)
UniSam
VL-LTR 2022
77.8
RosNet-50
Cap2Ang 2023
78.5
67.7
51.9
70.9
GaNTe Ada
79.2
Work
P 2012
uorup.
842200.7
Woak
Meta- 2023
uesup.
696:04
86.22033
VIT-B
Wook
UntSam + dist 2022
momp.
096400
00.5
Method
Many
Med.
Few
Overall Ace
Wook
UniSiam 2022
005+0.7
84.7+0.5
Slasin
UniSlam 2122
665+0.77
84.5
VIT 2021
50.5
28.5
6.9
31.6
CutMix 2015u
UniSlam 2112
masup.
66021117
MAE 2022a
74.7
48.2
19.4
54.5
Mistip p 2018
UniStam 2022
mesup..
66.110.5
80.120.8
DefT 2022
70.4
40.9
12.8
48.4
Ing2ingg 2002
UniStam 2022
mmmp..
678.0.77
$5.310.5
LVV 2023
73.6
56.4
41.0
60.9
Teglting "f 20022
UniStam 2022
72.4:0.7
0..7
LVT
Tag2Ting
74.3
56.4
34.3
60.5
The2 122000
UniSaam 2022
moopp..
+7.1+066
0..4
LVT
ing22nng
73.8
56.4
45.3
61.6
GuNTe (Ouns)
GaNTw-kdx Add (Qurs)
DAF Flocos 2024
UniSiam 2022
UniSiam 2022
UniSlam 2022
momp.
mssup.
moup.
78.8-0.0
78.0:0.6
06.5+2.2
88.0:10.4
88.0--0.6
84.8=1.4
LVV
LVT
Twe2Ting
GelTe Ada
74.9
74.0
56.9
55.6
48.3
62.2
the source images from other classes, we use a confusion matrix on the training data to find the top-4 most confused classes and only consider those classes for random sampling of the source image.
Augmenting training data with GeNIe-Ada improves accuracy on the "Few" set by 11.7% and 4.4% compared with LVTT only and LVTT with Txt2Img augmentation baselines respectively. In ResNet50, GeNIe-Ada outperforms Cap2Aug baseline in "Few" categories by 7.6%. The results are summarized in Table 4. Please refer to Section A.11 for implementation details.
Results:
4.4 Ablation and Further Analysis
Semantic Shift from Source to Target Class, The core motivation behind GeNIe-Ada is that by varying the noise ratio from 0 to 1, augmented sample X, will progressively shift its semantic category from source (5) in the beginning to target category (T) towards the end. However, somewhere between 0 and 1, X, will undergo a rapid transition from S to T. To remonst this hypothesis empirically, in Figs. 5 and A7, we visualize pairs of source images and target categories with their respective GeNIe generated augmentations for different noise ratios r, along with their corresponding PCA-projected embedding scatter plots (on the far left). We extract embeddings for all the images using a DINOy2 ViT-G pretrained backbone, which we assume as oracle model in identifying the right category. We observe that as r increases from 0.3 to 0.8, the images transition to embody more of the target category's 's semantics while preserving the contextual features of the source image. This transition of semantics can also be observed in the embedding plots (on the left) where they consistently shift. from the proximity of the source image (blue star) to the target class's centroid (red cross) as the noise ratio r incrcascs. The sparse distribution of points within the first and second images and r [0.2, 0.4] for the third image aligus with our intuition of rapid transition from category S to T, thus empirically affirming our motivation behind GeNIe-Ada. This semantic shift also corroborates with the insight provided in (Sclocchi et al., 2025) that at denoising timesteps beyond the srngedddttanitioned, but the generated sample may still be composed
[0.4, 0.6] for
transition, the class has completely of low-level elements of the initial image.
To further establish this, in Fig. 6, we demonstrate the efficacy of GeNIe in generating hard negatives at the decision boundaries of an SVM classifier, which is trained on the labelled support set of the few-shot tasks of mini-Tmagenet, without any augmentations. We then plot source and target class probabilities (P(Ys)X,)
10

---

Published in Transactions on Machine Learning Research (04/2025)
ratto
Torer centval image
Prompt A Ap Volcano.
Soarce Image
r=24
twS
H+0.7
ArompeAphoto Aphoto Mustroom
layet
Soane Image
m27
m0
Goldon Ratrever
Clem: fambter 'Trget Golden Nett
Figure Embedding visualizations of generative augmentations: We pass all generative augmentations through DINOv2 ViT-G (serving as an oracle) to extract their corresponding embeddings and visualize them with PCA. As shown, extent of semantic shifts varies based on both the source image and the target class.
GeNIn(r = 0.5)
GekTefr 0.6)
GeNIe(( 0.7)
GahInlr = (.8)
GeNIc- =x")
175
145
PIYSIX.) POLIX.)
Density
18
123
0.5
64
030
Probability
Probability
14
Probability
Probability
Protability
Figure 6: Why GeNIe augmentations are challenging? While deciding which class the generated augmentations (Xr) belong to is already difficult within r [0.6, 0.7] (due to high overlap between P(Ys(X,) and P(Y)(X.)). GeNIe-AGe Ada selects the best noise threshold (r*) offering the hardest negative sample.
and P(Yr|X,), respectively) of the generated augmentation samples X,.. For both 0.6 and 0.7, there is significant overlap between P(Ys(X,) and P(Y+|X,), making it difficult for the classifier to decide the correct class. On the right-band-side, Gda automatically selects the best r resulting in the most overlap between the two distributions, thus offering the hardest negative sample among the considered r values (for more details see A.1). Note that a large overlap between distributions not sufficient to call the generated samples hard negatives because they should also belong to the target category. This is, however, confirmed by the high Oracle accuracy in Table (elaborated in detail in the following paragraph) which verifies that majority of the generated augmentation samples do belong to the target category.
Label consistency of the generated samples. The choice of noise ratio r is important in producing hard negative examples. In Table 5, we present the accuracy of the GeNIe model across various noise ratios, alongside the oracle accuracy, which is an ImageNet pre-trained DeiT-Base (Touvron et al., 2021b) classifier. We observe decline the label consistency of generated data (quantified by the performance of the oracle model) when decreasing the noise level. Roducing r also results in degradation the performance of the final few-shot model (87.2% 77.6%) corroborating that an appropriate choice of plays a crucial role. We investigate this further in the following paragraph.
Effect of Noise in GeNIe. We examine the impact of noise on the performance of the few-shot model in Table 5. Noise levels r E |0.7, 0.8] yield the best performance. Conversely, utilizing noise levels below 0.7 diminishes performance due to label inconsistency, as is demonstrated in Table and Fig 5. As such, determining the appropriate noise level pivotal for the performance of GeNIe be able to generate challenging hard negatives while maintaining label consistency. An alternative approach finding the optimal noise level involves using Tda to adaptively select the noise level for each source image and
11

---

Published in Transactions on Machine Learning Research (04/2025)
Table 5: Effect of Noise and Difflusion Models in GeNIe: We use the same setting as in Table 1 to study the effect of the amount of noise. As expected (also shown in Fig 5), small noise results in worse accuracy since some generated images may be from the source category rather than the target one. For r = 0.5 only 73% of the generated data is from the target category. This behaviour is also shown in Fig. 2. Notably, reducing the noise level below 0.7 is associated with a decline in oracle accuracy and subsequent degradation in the performance of the final few-shot model. Note that the high oracle accuracy of GeNIe-Ada demonstrales its capability to adaplively select the noise level per source and target, ensuring semantic consistency with the intended target. To further demonstrate GeNIe's ability to generalize across different diffusion models, replace the diffusion model with SD3 and SDXT--uuro. The resulting accuracies follow similar trend to those in Table 1, confirming GeNIe's advantage over Txt2Ing across various diffusion models.
Method
Generative Model
Noise
ResNet-18
ResNett3d
ResNet-50
Oracle
1-shot
5-shot
1-shot
5-shot
1-shot
5-shot
Acc
Txt2Ing
SD 1.5
74.110.6
84.620.5
75.440.6
85.520.5
76.420.6
86.540.44
GellTe
SD 1.5
0.5
6..4
7..1
6..0+
7..8++0.6
6..7+0.9
77.6+0..6
0.5
GellIe
SD 1.5
0.6
69.7.10.77
80.710..5
71.110.7
82.2.10.5
72.110.7
82.8
85.8.10.
GellIe
SD 1.5
0.7
74.5+0066
83.3+0.5
76.4+0.6
84.4=0.5
07.1
85.0+0.4
94.
GellIe
SD 1.5
0.8
75.5+00.6
85.4+0.4
77.1+0066
86.3+0.4
77.3+006
87.2+0.4
98.2+0.1
CeNIe
SD 1.5
0.9
75.0100.6
85.310.4
77.610.6
86,21044
77.710.6
87.010.4
99.310.1
CeNIe-AAa
SD 1.5
Adaptive
76.820.6
85.910.4
78.510.6
86.620.4
78.620.6
87.940.4
98.910.2
Txt22Ing
SDXL-Turbo
72.5.10.3
82.110.6
76.2.10.2
84.41033
76.710..6
85.91.0.5
GellIe
SDXI-Turbo
0.5
61.2+0.5
73.5+0.2
61.5+0.2
74.9=0.3
63.1+002
76.5+00.6
GelIIe
SDXI-TTurb
0.6
70.2+002
79.3+0.4
71.2+0.7
81.4-0.6
73.2+0.2
82.4+0.5
GellIe
SDXL-Turbo
0.7
73.1100.3
83.510.5
76.110.6
85.3104
77.210.66
84.210.4
GellIe
SDXL-Ilubo
0.8
74.210.3
85.120.3
76.910.4
85.5
78.710.6
87.740.4
GellIe
SDXT-TTurbo
0.9
73.9+-0.4
84.9+0.7
76.6+0.77
84.2+0066
78.1+00.5
87.0+0.4
GeNIe-Ada
SDXL-TTurbo
Adaptive
76.1+0.3
87.1
78,9+0.5
85.2=0.5
79.0=0.66
88.6+0.2
Txt2Ing
SD 3
73.6+1.7
1.29+1.2
76.7+1.5
85.5+13
17.2+1.9
85.0+1.2
GelIIe
SD 3
62.01112
72.911.1
62.510.9
73.9110
64.110.5
76.111.9
GellIe
SD
70.811.5
79.141.9
71.41.2
82.1113
74.1+1.5
83.41
GeliTe
SD 3
0.7
74.6+00.8
8..
76.5+1.9
86,2+1.6
78.5+1.9
84.0 1.1
GellIe
SD 3
0.8
75.9+112
8..321.7
77.821.9
85.5
79.2+1.7
88.3+1.9
GellIe
SD 3
0.9
75.1+0.55
1.
78.1+1.3
86,2+1.2
77.1+1.9
88.9+-0.8
CaNlIe-Ada
SD 3
Adaptive
76.8.11.
87.5.1.55
78,911.3
87.7
79.111.4
89.5.11.0
target class. As demonstrated in Tables and 2, GeNIe-Ada matches or outperforms GeNIe with fixed noise levels.
Effect of Diffusion Models in GeNIe. We have tried experimenting with both smaller well more recent diffusion models. More specifically, we have used Stable Diffusion XL-Turbo to generate hard-negatives through GeNIe and GeNIe-AGe Ada. Few-shot classification results on minilmagenet with these augmentations are shown in Table 5. The accuracies follow a similar trend to that of Table 1, where Stable Diffusion 1.5 Ada improves UniSiam's few-shot performance the most as
was used to generate augmentations. compared to GeNIe with different. noise ratios v, and even when compared to Txt2Img. This empirically indicates the robustness of GeNIe and GeNIe-Ada to different diffusion engines. Note that, Stable Diffusion XL-Turbo by default uses 4 steps for the sake of optimization, and to ensure we can have the right granularity for the choice of we have set the number of steps to 10. That is already 5 times faster than the standard Stable Diffusion v1.5 with 50 steps. Our experiments with Stable Diffusion v3 (which is a totally different model with a Transformers backbone) also in Table 5 also convey the same message. As such, we believe our approach is generalizable across different diffusion models.
GeNIe's applicability vs. VLMs and Large Scale Pro-Trained Backbones. GeNTe enables down- stream task performance improvements by fine-tuning the pre-trained backbone using our challenging hard- negative samples. This GeNle fine-tuned backbone could then be used for various different test samples inference time. This allows the user to benefit from performance improvements, while using computation- ally cheap pre-trained backbone (such as ResNet-18. 18,3,,,0, as opposed to a compute and memory intensive pre-trained VLMs. Using VLMs is even more challenging in compule-constrainee edge-device applications, where only a few gigabytes of memory is available for model inference upon deployment. That being said, we include comparisons against LAION (Schuhmann et al., 2022) vasion-language pre-trained CLIP (Rad- ford et al., 2021) ResNet50 in Appendix A.5, Table A3 to evaluate if GeNIe can still render performance improvements using hard negative samples. We observe that while this new CLIP backbone's performance
12

---

Published in Transactions on Machine Learning Research (04/2025)
appears saturated on this task ( 93%% 1-shot accuracy), GeNIe-Ada still achieves 1.3%, 0.5% improvements in the 1-shot, 5-shot settings, respectively, thus corroborating the efficacy of our hard-negative samples in data-deficient settings.
5 Concluding Remarks
GeNIe, for the first time to our knowledge, combines contradictory sources of information (a source image, and a different target category prompt) through a noise adjustment strategy into a conditional latent diffusion model to generate challenging augmentations, which can serve as hard negatives.
Limitation. The required time to create augmentations through GeNIe on par with any typical diffusionn based competitors (Azizi et al., 2023; He et al., 2022b); however, this is naturally slower than traditional augmentation techniques (Yun et al., 2019a; Zhang et al., 2018). This is not bottleneck in offline augmen- tation strategies, but can be considered a limiting factor in real-time scenarios. Recent studies are already mitigating this through advancements in diffusion model efficiency (Sauer et al., 2023; Meng et al., 2023; Liu et al., 2023). Another challenge present in any generative Al-based augmentation technique the do- main shift between the distribution of training data and the downstream context they might be used for augmentation. A possible remedy is to fine-tune the diffusion backbone on rather small dataset from the downstream task.
Broader Impact. GeNIe can have significant impact when it comes to generating challenging augmenta- tions and thus enhancing downstream tasks beyond classification. Like any other generative model, GeNIe can also introduce inherent biases stemming from the training data used to build its diffusion backbone, which can reflect and amplify societal prejudices or inaccuracies. Therefore, it crucial to carefully mitigate potential biases in generative models such as GeNIe ensure fair and ethical deployment of deep learning systems.
References
Arman Afrasiyabi, Jean-France Lalonde, and Christian Gagné. Associative alignment for few-shot image classification. In ECCV, 2019.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katic Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Scrkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, An- drew Zisserman, and Karen Simonyan. Flamingo: visual language model for few-shot learning, 2022.
Antreas Antoniou and Amos Storkey. Assume, augment and learn: Unsupervised few-shot meta-learning via random labels and data augmentation. arzic:1902. 09884, 2019.
Shekoofeh Azizi, Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David J. Fleet. Synthetic data from diffusion models improves imagenet classification, 2023.
Lukas Bossard, Matthien Guillaumin, and Luc Van Gool. Food-101 - mining discriminative components with random forests. In European Conference on Compuler Vision, 2014.
Jiarui Cai, Yizhou Wang, Jenq-Neng Hwang, et al. Ace: Ally complementary experts for solving long-tailed recognition in one-shot. In ICCV, pp. 112-121, 2021.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distributionnavaaa margin loss. NewrIPS, 32, 2019.
Atoosa Chegini and Sohcil Feizi. Identifying and mitigating model failures through few-shot clip-aided diffusion generation. arXiv preprinl xxXim:2312.05464 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In ICML, 2020.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu- Yu-Chiang Frank Wang, and Jin-Bin Huang. A closer look few-shot classification. In ICLR, 2019a.
Wentao Chen, Chenyang Si, Wei Wang, Liang Wang, Zilei Wang, and Tieniu Tan. Few-shot learning with part discovery and augmentation from unlabeled images. arXiv preprint arXiv:2105. 11874, 2021.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, 2021.
Zhengyu Chen, Jixie Ge, Heshen Zhan, Siteng Huang, and Donglin Wang. Pareto self-supervised training for few-shot lcarning. In CVPR, 2021.
Zitian Chen, Yanwci Fu, Yu-Xiong Wang, Lin Ma, Wei Liu, and Martial Hebert. Image deformation meta- networks for one-shot learning. In CVPR, 2019b.
Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. Autoaugment: Learning augmentation policies from data, 2019a.
Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with reduced search space, 2019b.
Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoc Lc. Randaugment: Practical automated data augmentation with a reduced search space. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Bal- can, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 18613- 18624. Curran Associates, Inc., 2020. URL attps://proceedings nwwrips. http://ig....uuee....cc///////// Papeef-appr.... pdf.
18866ee000b114440377777686000000
Jicquan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In ICCV, pp. 715-724, 2021a.
Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 715-724, 2021b.
Jiequan Cui, Shu Liu, Zhuotao Tian, Zhisheng Zhong, and Jiaya Jia. Reslt: Residual learning for long-tailed
recognition. IEEE transactions on pallern analysis and machine intelligence, 35(3) 3706, 2022.
Yin Cui, Menglin Jia, Tsung-Y Yi Lin, Yang Song, and Serge Belongic. Class-balanced loss based on effective number of samples. In CVPR, pp. 9268 0087,927,,77 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248 255. leee, 2009.
Guncot S. Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. bascline for few-shot image classification. In ICLR, 2020.
Mucong Ding, Bang An, Yuancheng Xu, Anirudh Satheesh, and Furong Huang. SAFLEX: Self-adaptive augmentation via feature label extrapolation. In The Twelfth International Conference Learning Rep- resentations, 2024. URL
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
Lisa Dunlap, Clara Mohri, Han Zhang, Devin Guillory, Trevor Darrell, Joseph F. Gonzalez, Anna Rohrbach, and Aditi Raghunathan. Using language to extend to unseen domains. International Conference on Learning Representations (ICLR), 2023a.
Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang, Joseph E. Gonzalez, and Trevor Darrell. Diversify your vision datasets with automatic diffusion-based augmentation, 2023b.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Nikita Dvornik, Julien Mairal, and Cordelia Schmid. Diversity with cooperation: Ensemble methods for few-shot classification. In TCCV, 2019.
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning, 2023.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic mela-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference Machine Learning, pp- 1126-1135, 2017.
Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, and Hayit Greenspan. Gan based synthetic medical image angmentation for increased performance in liver lesion classification. Newrocomputingg 2018.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashník, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth word: Personalizing text-to-image generation using textual inversion. arXiv preprint Xrx::22008.000 2022a.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to-image generation using textual inversion, 2022b. 2tt.s///20.0011.....
URL
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to-image generation using textual inversion, 2022c.
URL
Alexandros Graikos, Srikar Yellapragada, and Dimitris Samaras. Conditional generation from pre-trained diffusion models using denoiser representations. In 34th British Machine Vision Conference 2023, BMVC 2023, Aberdeen, UK, November 20-24, 2023. BMVA, 2023a. URL https::/papont mmvc2023.org/0477 pdf.
Alexandros Graikos, Srikar Yellapragada, and Dimitris Samaras. Conditional generation from unconditional diffusion models using denoiser representations. arXie preprint arXin:2306.011 019000, 2023b.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners, 2021.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. In CVPR, pp. 15979-15988. TEEE, 2022a.
Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and Xiaojuan Qi. Is synthetic data from generative models ready for image recognition? arXiv preprint arXim:2210.07574, 2022b.
Reyhane Askari Hemmat, Mohammad Pezeshki, Florian Bordes, Michal Drozdzal, and Adriana Romero- Soriano. Feedback-guided data synthesis for imbalanced classification, 2023.
Dan Hendryeks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. AugMix: A simple data processing method to improve robustness and uncertainty. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Yan Hong, Jianfu Zhang, Zhongyi Sun, and Ke Yan. Safa: Sample-adaptive feature augmentation for long-tailed image classification. In ECCV, 2022.
Kyle Hsu, Sergey Levine, and Chelsea Finn. Unsupervised learning via meta-learning. In ICLR, 2018.
Wentao Hu, Xiurong Jiang, Jiarun Liu, Yuqi Yang, and Hui Tian. Meta-dm: Applications of diffusion models on few-shot learning, 2023.
Sheng- Wei Huang, Cho-Tsung Lin, Shu-Ping Chen, Yen-Yi Yi Wu an Po-Hao Hsu, and Shang- Lai. Auggan: Cross domain adaptation with gan-based data augmentation. European Conference on Computer Vision, 2018.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. Distilling model failures as directions in latent space, 2022.
Huiwon Jang, Hankook Lee, and Jinwoo Shin. Unsupervised meta-learning via few-shot pseudo-supervised contrastive learning. In The Eleventh International Conference on Learning Representations, 2022.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalan- tidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1900.000 092177 2019.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalan- tidis. Decoupling representation and classifier for long-tailed recognition. In JCLR, 2020.
Siavash Khodadadeh, Ladislau Boloni, and Mubarak Shah. Unsupervised meta-learning for few-shot image classification. In NewrIPS, 2019.
Jang-Hyun Kim, Wonho Choo, and Hyun Oh Song. Puzzle mix: Exploiting saliency and local statistics for optimal mixup. In International Conference on Machine Learning, pp. 5275-5285. PMLR, 2020.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3D object representations for fine-grained cate- gorization. In Workshop on 3D Representation and Recognition, Sydney, Australia, 2013.
Alexander C. Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier, 2023.
Bolian Li, Zongbo Han, Haining Li, Huazhu Fu, and Changqing Zhang. Trustworthy long-tailed classification. In CVPR, pp. 6970-6979, 2022a.
Daiqing Li, Huan Ling, Seung Wook Kim, Karsten Kreis, Adela Barriuso, Sanja Fidler, and Antonio Torralba, Bigdatasotgan: Synthesizing imagenet with pixel-wise annotations, 2022b.
Jun Li, Zichang Tan, Jun Wan, Zhen Lei, and Guodong Guo. Nested collaborative learning for long-tailed visual recognition. In CVPR, pp. 6949 6958, 2022c.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified dision-language understanding and generation, 2022d.
Kai Li, Yulun Zhang, Kumpeng Li, and Yun Fu. Adversarial feature hallucination networks for few-shot learning. In CVPR, 2020.
Mengke Li, Yiu-ming Choung, Yang Lu, et al. Long-tailed visual recognition via gaussian clouded logit adjustment. In CVPR, pp. 6929 6938, 2022e.
Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio S Feris, Piotr Indyk, and Dina Katabi. Targeted supervised contrastive learning for taiget recognition. In CVPR, pp. 6918 0928, 2022f.
Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang, Mingsheng Long, and Han Hu. Negative margin matters: Undorstanding margin in few-shot classification. In ECCV, 2020.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One stop enough for high-quality liffusion-based lext-to-image generation. In The Twelfth International Conference on Learning Represen- tations, 2023.
Zicheng Liu, Siyuan Li, Di Wu, Zihan Liu, Zhiyuan Chen, Lirong Wu, and Stan Z Li. Automix: Unveiling the power of mixup for stronger classifiers. In Computer Vision ECCV 2022: 17th European Conference, Tel Avio Israel, October 23-27, 2022, Proceedings, Part XXIV, pp. 441-458. Springer, 2022.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X. Yu. Large-scale long- tailed recognition in an open world. In CVPR, 2019.
16

---

Published in Transactions on Machine Learning Research (04/2025)
Yuning Lu, Liangjian Wen, Jianzhuang Liu, Yajing Liu, and Xinmei Tian. Self-supervision can be good few-shot learner. In European Conference Computer Vision, pp. 740-758. Springer, 2022.
Xue-Jing Luo, Shuo Wang, Zongwei Wu, Christos Sakaridis, Yun Cheng, Deng-Ping Fan, and Luc Van Gool. Camdiff: Camouflage image augmentation via diffusion model, 2023.
Lorenzo Luzi, Ali Siahkoohi, Paul M Mayer, Josue Cadco-Rod and Richard Baraniuk. Boomerang: Local sampling on image manifolds using diffusion models, 2022.
Avery Ma, Nikita Dvorník, Ran Zhang, Leila Pishdad, Konstantinos Derpanis, and Afsaneh Fazly, Sage: Saliency-guided mixup with optimal rearrangements. orXiv preprint arXiv:2211.0011, 2022.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint orXiv:136..52 2013.
Jiayuan Mao, Tete Xiao, Yuning Jiang, and Zhimin Cao. What can help pedestrian detection?, 2017.
Carlos Medina, Arnout Devos, and Matthias Grossglauser. Self-supervised prototypical transfer learning for few-shot classification. In ICMLW, 2020.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and oditing with stochastic differential equations. ar preprint or Xrxi:2108.00107, 2021.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14297 14306, 2023.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In ICLR, 2021.
Maxime Oquab, Timothée Darcet, Th6o Moutakanni, Huy Vo, Mare Szafranice, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Woj- ciech Galuba, Russell Howes, Yao Huang, Shang- Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bo- janowski. Dinov2: Learning robust visual features without supervision, 2023.
William Peebles, Jun-Yan Zhu, Richard Zhang, Antonio Torralba, Alexei Efros, and Eli Shechtman. Gan- supervised dense visual alignment. In CVPR, 2022.
Suzanne Petryk, Lisa Dunlap, Keyan Nasseri, Joseph Gonzalez, Trevor Darrell, and Anna. Rohrbach. On guiding visual attention with language specification. In Conference Computer Vision and Pattern Recognition (CVPR), 2022. doi: 10.48550/ 22015//A2AA...2220022222222 URL 08926.
org/ abs/2202.
Viraj Prabhu, Sriram Yenamandra, Prithvijit Chattopadhyay, and Judy Hoffman. Lance: Stress- visual models by generating language-guided counterfactual images. Advances in Neural Information Processing Systems, 36, 2024.
Siyuan Qiao, Chenxi Liu, Wci Shen, and Alan Yuille. Few-shot image recognition by predicting paramoters from activations. In CVPR, 2018.
Tiexin Qin, Wenbin Li, Yinghuan Shi, and Gao Yang. Unsupervised few-shot learning via distribution shift-based augmentation. arziv:2004.55805 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkiin Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763. PMLR, 2021.
17

---

Published in Transactions on Machine Learning Research (04/2025)
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Nya Sutskever. Zero-shot text-lo-image generation. In International Conference on Machine Learning, pp. 8821 8831. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.0614 2022.
Sachin Ravi and H. Larochelle. Optimization as model for few-shot learning. In ICLR, 2017.
Mengye Ren, Sachin Ravi, Eleni Triantafillou, Jake Snell, Kevin Swersky, Josh B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. In In- ternalional Conference Learning Representations, 2018. URL attps://openrewee http::////ppareieeeeeetettrrttt HJcSzz-CZ.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021a.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resohttin image synthesis with latent diffusion models, 2021b.
Aniket Roy, Anshul Shah, Ketul Shah, Anirban Roy, and Rama Chellappa. Cap2aug: Caption guided image image data augmentation, 2023.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorcalistic text- to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479 04::4479-36494, 2022.
Dvir Samuel and Gal Chechik. Distributional robustness loss for long-tail learning. In ICCV, 2021.
Swami Sankaranarayanan, Yogesh Balaji, Carlos Domingo Castillo, and Rama Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint ar arXtc:2311.100.. 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kun- durthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsov. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
Antonio Sclocchi, Alessandro Favero, and Matthicu Wyart. A phase transition in diffusion models reveals the hierarchical nature of data, 2024.
Antonio Sclocchi, Alessandro Favero, and Matthieu Wyart. A phase transition in diffusion models reveals the hierarchical nature of data. Proceedings of the National Academy of Sciences, 124(1)1124, 2025.
Viktoriia Sharmanska, Lisa Anne Hendricks, Trevor Darrell, and Novi Quadrianto. Contrastive examples for addressing the lyranny of the majority. CoRR, abs/ abs/2004..06 20044.06444 2020. URL httpa://arxiv.orr//an 2004.06524.
Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, and Clinton Fookes. Boosting zero-shot classification with synthetic data diversity via stable diffusion. arXiv preprint arXin:2302.08208 2023.
Ojas Kishorkumar Shirckar, Anuj Singh, and Hadi Jamali-Rad. Self-attention message passing for contrastive few-shot learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 54266 5436, January 2023.
18

---

Published in Transactions on Machine Learning Research (04/2025)
Connor Shorten and Taghi M Khoshgoftaar A survey image data augmentation for deep learning. Journal of big dala, 6(1):1-48, 2019.
Anuj Rajeeva Singh and Hadi Jamali-Rad. Transductive decoupled variational inference for few-shot
classification. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https: / /
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, 2017.
Jong-Chyi Su, Subhransu Maji, and Bharath Hariharan. When does self-supervision improve few-shot learning? In ECCV, 2020.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning. In CVPR, 2018.
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. Long-tailed classification by keeping the good and removing the bad momentum causal effect. NewrIPS, 53:1513-1524, 2020.
Kowshik Thopalli, Rakshith Subramanyam, Pavan Turaga, and Jayaraman J. Thiagarajan. Target-aware generalive augmentations for single-shot adaptation. In Proceedings of the 40th International Conference on Machine Learning, ICML 233. JMLR.org, 2023.
Changyao Tian, Wenhai Wang, Xizhou Zhu, Jifeng Dai, and Yu Qiao. Vi-Itr: Learning class-wise visual- linguistic representation for long-tailed visual recognition. In ECCV 2022, 2022.
Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. Stablerep: Synthetic images from text-to-inage models make strong visual representation learners, 2023. URL https://arxiv.org/ abs/2306. 00984.
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Mochine Learning (ICML), 2021a.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers and distillation through attention, 2021b.
Hugo Touvron, Matthieu Cord, and Hervé Jégou. Deit iii: Revenge of the vit. In ECCV, 2022.
Brandon Trabucco, Kyle Doherty, Max A Gurinas, and Ruslan Salakhutdinov. Effective data augmentation with diffusion models. In The Twelfth International Conference on Learning Represendations, 2024. URL httpss://pperrevier. pest/foorumidd7///uAAzz
Nontawat. Tritrong, Pitchaporn Rewathowornwong, and Supasorn Suwajanakorn. Repurposing gans for one-shot semantic part segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongic. The caltech-ucsd birds- 200-2011 dataset, 2011.
Haoqing Wang and Zhi-Hong Deng. Contrastive prototypical network with wasscrstein confidence penalty. In European Conference on Computer Vision, pp. 665 682. Springer, 2022.
Hualiang Wang, Siming Fu, Xiaoxuan He, Hangxiang Fang, Zuozhu Liu, and Haoji Hu. Towards calibrated hyper-sphere representation via distribution overlap coefficient for long-tailed learning. In ECCV, 2022.
Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X. Yu. Long-tailed recognition by routing diverse distribution-aware experts. In ICLR. OpenReview.net net, 2021.
Yue Xu, Yong-Lu Lu Li, Jiefeng Li, and Cewn Lu. Constructing balance from imbalance for long-tailed image recognition. In ECCV, pp. 38 56. Springer, 2022.
19

---

Published in Transactions on Machine Learning Research (04/2025)
Zhengzhuo Xu, Ruikang Liu, Shuo Yang, Zenghao Chai, and Chun Yuan. Learning imbalanced data with vision transformers, 2023.
Hong Xuan, Abby Stylianou, Xiaotong Liu, and Robert Pless, Hard negative examples are hard, but useful, 2021.
Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In CVPR, 2020.
Sangdoo Yun, Dongyoon Han, Scong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In 1CCV, pp. 6023 6032, 2019a.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In International Conference on Compuler Vision (ICCV), 2019b.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.
Songyang Zhang, Zeming Li, Shipeng Yan, Xuming He, and Jian Sun. Distribution alignment: A unified framework for long-tail visual recognition. In CVPR, pp. 1361-23700 2021a.
Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. Test-agnostic long-tailed recognition by test-time aggregating diverse experts with self-supervision. arXiv preprint arXim:2107.092. 2021b.
Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Tean-Francois Lafleche, Adela Barriuso, Antonio Tor- ralba, and Sanja Fidler. Datasetgan: Efficient labeled data factory with minimal human effort. In CVPR, 2021c.
Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long-tailed recognition. CVPR, pp. 16489 16498. Computer Vision Foundation IEEE, 2021.
Ziqi Zhou, Xi Qiu, Jiangtao Xie, Jianan Wu, and Chi Zhang, Binocular mutual learning for improving few-shot classification. In ICCV, 2021.
Jianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen, and Yu-Gar Gang Jiang, Balanced contrastive learning for long-tailed visual recognition. In CVPR, pp. 6908-6917, 2022.
20

---

Published in Transactions on Machine Learning Research (04/2025)
A Appendix
A.1 Analyzing GeNIe, GeNIe-Ada's Class-Probabiilttee
The core aim of GeNIe and GeNIe-Ada is address the failure modes of classifier by generating chal- lenging samples located near the decision boundary of each class pair, which facilitates the learning pro- cess in effectively enhancing the decision boundary between classes. As summarized in Table 5 and il- lustrated in Fig. 5, we have empirically corroborated that GeNIe and GeNIe-Ada can respectively pro- duce samples Xp- that are negative with respect to the source image Xs, while semantically be- longing to the class T. To further analyze the effectiveness of GeNIe and GeNIe-Ada, we compare the
GeNIa(r 0.5)
GeliTefr 0.6)
GeNIe(y 0.7)
Gellafy - 0.8)
GeNIe(y 0.9)
POG(X.)
33-
Density
L2
26-
32
Probability
Probability
Probability
Probabilisy
Probauiaty
Figure Al: P(Y)(Xp) and P(Yr[X,) for E (0.5, 0.6, 0.7, 0.8, 0.9). On average, the classifier confidently predicts the source class more than the target class for X, for r = 0.5, and vice-versa for r= 0.8, 0.9. However, for 0.6, 0.7, the classifier struggles to classify X,, indicating that the augmented samples are located closer to the decision boundary.
source class- plssa-robabilltiis P(Ys) X,) and target-class probabilities P(Ys(X,) of augmented samples X To compute these class probabilities, we first fit SVM classifier (as followed in UniSiam (Lu et al., 2022)) only the labelled support set embeddings of each episode in the miniImagenet test dataset. Then, we perform inference us- ing each episode's SVM classifier on its respective X,'s and extract its class probabilities of belonging to its source class S and larget class T. These per target class probabilities are then averaged for each episode for each (0.5, 0.6, 0.7, 0.8, 0.9} in the case of GeNIe and for the
GeNIe-Ada("' = r*)
2.00
175
P(Y)|Xp)) P(G)XP))
source and
Density
125
1.00
optimal = r* per sample in the case of GeNIe-AAa, plotted as density plots in Fig. A1, Fig. A2, respectively. Fig. A1 illus- trates that P('s (()) and ((Yr|X,) have significant overlap in the case of E {0.6, 0.7} indicating class-confusion for X,.
0.75
025
Furthermore, Fig. A2 illustrates that when using the opti- mal r" found by GeNIe-Ada per sample, P(Ys|Xp) and P(Y significantly overlap around probability scores of 0.2 - 0045, indicating class confusion for GeNIe-Ada augmenta- tions. This corroborates with our analysis in Section 4.4, Ta- ble 5 and additionally empirically proves that the augmented samples generated by GeNIe for {0.6,0.7 and GeNIe-Ada for = r" are actually located near the decision boundary of each class pair.
0.00
Probability
Figure A2: Significant overlap between
P(Ys ) and P(Yr|X,-) indicates high class- confusion for augmented samples generated by CeNIe-Ada.
A.2 Independence of Generated Augmentations from Downstream Test Sets
Here we analyzed whether the augmented samples generated by GeNIe using the diffusion model overlap with the test set of the downstream task. To set the stage, we extracted the latent embeddings corresponding to the train set (i.c., support), test set (i.c., query), and augmentations generated by GeNIe. Fig A3 illustrates the distribution of distances between train-test and augmentation-tess pairs across 600 episodes. Notably, the mean distance of augmentation-test pairs is higher than that of train-test. pairs, indicating that the
21

---

Published in Transactions on Machine Learning Research (04/2025)
Train 4
Figure A3: Comparison of embedding distributions and UMAP visualization for train, test, and GeNle-angmented samples.
augmented samples are distinet from the test set. This observation aligns with the fundamental assumption of train and test sets being mutually exclusive. Additionally, Fig A3 provides further evidence through a UMAP embedding plot of a randomly selected episode, where the embeddings of train, test, and augmented samples visualized. The plot reveals clear separations between the test set and augmented samples, further confirming that the augmented samples do not overlap with resemble the test set in embedding space. These findings validate that the diillusion-generateed augnentatiins are independent of the downstream task's test set, ensuring the integrity of the evaluation process.
A.3 Computational Complexity of GeNIe and GeNIe-Ada
Here we provide details on the computational com- plexity GeNIe across multiple noising ratios r and GeNIe-Ada when operating on a search space of r [0.6, 0.8). Computational complexity has been reported in terms of the total number of inference/ inference/denoisingg: diffoisinnddiffuiioo steps and the runtime in seconds per generated image. The runtime has been averaged over 10 different sooge-generationn on an NVIDIA Tesla-V V100 GPU with 16GB VRAM with steps of denoising using DPM schod- uler with StableDiffusion v1.5. As can be seen in Tab. A1, GeNIe is approximately 1/r times faster than the base diffussin model (referred to the Txt2Ing augmentation baseline). This empirically corroborates with the total number of denoising steps using in GeNIe vs. Txt2Img. Since, GeNIe-AGe Ada scans for the best hard-negative in E [..6,088) it incurs a computa- tional cost of = 2.2x the Txt2Img. Note that the runtime for Ada reported in Tab. Al also includes the runtime of performing a batched forward pass through a ResNet-50 feature extraction backbone.
Table A1: Computational Complexity
Augmentation
Inf. Steps
Runtime [sec/img:
Txt2Ing
T
4.12
(eN
0.5 x T
2.17
GeNIe(rr0.6)
0.6 x T
2.59
GeNIe(r=0.77
0.7 x
2.98
GeNTe(r=0.8)
0.8 x
3.46
GeNIe-Ada
2.1 x T
9.22
A.4 Additional Augmentation Comparisons
We compute few-shot classification scores on mini- Miaie-e with additional combinations of traditional augmentations. We introduce a Mixed augmentation scheme where we use a combination of Weak and Strong augmentations together. We also experiment the scenario where CutMix and MixUp are used alongside the Mixed augmentation strategy indicated by Mixed+CuttMis and Mixed+ MixUU. Finally, with combination of GeNIe along with MixUp, similar to (Graikos et al., 2023b). As can be seen in Tab. A2, we notice marginal improvements of upto 0.6% by using the Mixed augmentations either with or without the Mix, MixUp counterparts. We also notice drop in performance of upto 0.9% when MixUp used along with GeNIe. This follows the general trend of drop performance when using CutMix or MixUp, as reported in Tab. 1.
experiment
A.5 Comparison against large-scale pre-trained CLIP backbone
We evaluate GeNle and GeNIe-Ada with few-shot pre-trained ResNet-18.34.1 backbones to demonstrate that hard-negative augmentations can boost downstream-task performance of any data-deficient pre-
22

---

Published in Transactions on Machine Learning Research (04/2025)
Table A2: mini-Imaan mini-ImageNNe:; We augmentations (5-way, 1-shot) and (ā-way, 5-shot) few-shot settings of
mini-Imagenel dataset with 2 different backbones (ResNet-18 and 50). We compare with additional combinat ions of traditional augmentations, with and without GeNIe. The number of generated images per class is 4 for 1-shot and 20 for 5-shot settings.
BesNet-18
ReaNet-50
Augienntation
Method
Pre-training
S-ebos
Augnentation
Method
Pre-training
I-shot
Weak
UniSiam 2022
medp.
63. -1..1.0.8
81,410.55
Wook
CuiSiom 2022
unep.
616110.5
63.410.5
Strong
UsiSiom 2022
weup.
02 82..:
Strong 81.910.4
81,22:05
CaiSiom 2022
ussup
61520.
80.20..5
Mixed
UniSiam 2002
2...5
Mixedd
UniStam 8022
uncup.
64.500.5
80.800.5
CutMix 201
UaiSiam 2022
meup-
7
CusMix 201
CusSia 2022
umsp.
642400
502200.5
Up 2018
UniSium 2022
meup
62.1+00
80.7+0.6
MisUp 2018
UniSiam 2002
mosp-
63.8+0.8
64.600.5
Mised - MizUp 2018
UaiSium 2002
65.7+10.
82.1=0.2
Mixed-MMiUUp 2018
CuiSiam 2022
64.500.7
84.++0.7
CutMix
UniSism 2002
meep
61.9:0088
81:0::05
Miwed- CutMix 2018
UniSiam 2022
meep.
03.5+005
63.010.8
Rusion 2013
UsiSium 2022
moup
61.3+1.8
89.0+144
Disiio 2024
CuiSiam 2022
meup-
65.7+1.8
83.9+1.22
Gel MaxUp
UniSiam 2022
meup.
71.8.00.5
84.5.10.5
Geslle-MixUp
UuiSiam 2022
unrup.
76.11.0.5
65.0100.7
oatto (Ours)
UaiSiam 2022
75.020.6
85.4200.4
Guttee (Ours)
CuiStam 2022
unsu..
17.3:0.6
87.220.4
GolIe-Ada (Ours)
UsiSlam
meup.
76.8100.6
95.010.4
Golle-Ada (Ours)
UuiStam 2022
uncup.
78.6.0066
87.010.4
Irained backbones. In this subsection, we conduct an experiment where we use CLIP (Radford et al., 2021) pre-trained ResNet50 backbone rather than UniSiam pre-trained ResNet50. We then evaluate if GeNTe, GeNle-Ada can improve CLIP's few-shot accuracies to clarify whether the benefits of our approach come primarily from the generated negative samples or from the additional label information obtained by training on LAION. As can be seen in Table A3, we observe that while the CLIP backbone's performance appears saturated on this task 93% 1-shot accuracy), GeNle still achieves 1.3%, 0.5% improvements in the 1-shot, 5-shot settings, respectively.
Table A3: Performance comparison against CLIP pro-trained ResNet-50.
Augmentation
Method
Pre-training
I-shot
5-shot
Mixed
CLJP ResNet-50
unsup.
93.1-0.3
95.6+0022
GeNle
CLIP ResNei-50
ummp.
93.
95,8100.3
Gdale-Ada
CLIP ResNet-50
umsup.
91.1=0.2
96.110.2
A.6 Effect of Linear Classification Head vs. SVM
Following (Dhillon et al., 2020), we use an SVM classification head for few-shot and fine-grained datasets. However, our augmentation method is not restricted to this choice of classification head. In Tables A4, A5, we use a single layer linear classification (LL) head on top of the embedding extraction backbone, following (Dhillon ct al., 2020). We notice that GeNIe and GeNIe-Ada offer similar on-par performances with SVM and linear-layer (LL) classification heads for 1-shot and 5-shot minilmagenet classification tasks as well the fine-grained few-shot classification datasets. This corroborates GeNIe's 's robustness to both SVM and linear-layer fine-tuning based classification methods.
Table A4: We use our augmentations (5 (5-way, 1-shot) and (5-way, 5-shot) few-shot settings of mini- Imagenet dataset with 2 different backbones (ResNet-18 and 50) with a linear-layer (LL) and SVM classification head. The number of generated images per class is 4 for 1-shot and 20 for 5-shot settings.
ResNet-18
ResNett-0
Auguemtation
Mothod
Pre-trining
I-shot
I-shot
Augroentation
Mothod
Pre-tesining
I-shot
5-shot
GellTe +
UniSiam (2024)
unsup.
76.9+-0.
85.1
GellTw + L.
UniSiaun (2024)
unsu..
77.8-0.8
87.6=0.6
Galle--tas
UniSlam (2024)
unap..
76.9:00.5
85.7:0.6
Gellle-Ada L.L.
UniSiam (2021)
78.1-0.5
87.8-0.2
Swllla 5VM
UniSiam
usspp.
76.5++0.0
85.4-0..
SVM
UniSium (2024)
mssp.
77.3-0.6
87.2-0.4
SdMle-Ids + SVM (Ours)
UniSiam (2024)
unnp.
76.810.6
85.910.4
Gelile-Ada SVM
UniSiam (2021)
mnup.
78.6.006
87.0.0.4
A.7 Effect of Backbone for Noise Ratio Selector in GeNIe-AAd
To analyze the effect of the backbone feature extractor fa on selecting the optimal hard-negative using GeNIe-Ada, we use a pre-trained DeiT-B (Touvron et al., 2021a) instead of the UniSiam pretrained ResNet backbone. However, we still utilize the same ResNet backbone for few-shot classification. As shown Tab. A6, we notice a marginal improvement of upto 0.7% when using -eNIe-Ada+Delll as compared to GeNIe-Ada which uses the UniSiam pre-trained ResNet backbone. This suggests that there is still potential
23

---

Published in Transactions on Machine Learning Research (04/2025)
Table A5: Few-shot accuracies with linear-layer (LL) and SVM classification heads fine-grained datasets.
Method
Birds (CUB2000
Cars (Cars196)
Foods (Food101)
Aircraft
Baselin
90.3
19.8
82.9
29.2
Img2inng
90.7
50.4
87.4
31.0
Iug2ling #
91.3
36.4
91.7
34.7
Txt2ing
92.0
93.0
11.7
GeNIe (==..)
92.0
84.6
91.5
39.8
GeNIe (x=0.6)-SVM
92.2
87.1
92.5
15.0
GeNIe (==0.))
V2.5
87.9
92.9
47.0
GeNIe (r=0.M)SVM
92.5
87.7
93.1
46.5
GeNTe
[r 0.9)-SVM
92.4
87.1
93.1
45.7
GeNIe-Ad-SSM
92.6
87.9
93.1
16.9
GeNTe (r=0.5)-LL
92.4
84.1
91.4
GeNIe (r=0.6)LLL
91.8
87.5
92.1
44.7
GeNTe (r=0.7)-LI.
92.9
88.2
92.4
47.2
GeNIe (x=0.8)-LL
92.3
87.9
92.7
46.4
GeNIe (r=0.9)-LL
92.8
87.6
93.5
16.2
GeNTe-Ada-LL
92.1
87.4
93.7
46.3
to develop more effective strategics for selecting noise ratios to further enhance GeNIe. However, in this paper, we limit our exploration to GeNIe-Ada and leave these improvements for future work.
Table A6: Effect of Backbone for Noise Ratio Selector in GeNIe-AAa: We evaluate the impact of the noise ratio selector used in GeNIe-Ada (fa())) Note that in all experiments presented in this paper, we use the same backbone for fo(.) that is subsequently fine-tuned for few-shot classification tasks. However, to analyze the effect of fo(.) on sampled augmentations, we replace it with more powerful backbone, specifically DeiT-B pretrained on ImageNet- 1K. It is important to note that this is not a practical assumption: if Deil'-B were available for noise selection, it could also used as the classifier in few-shot experiments, outperforming the weaker backbones employed in our study. Nevertheless, this experiment demonstrates that using a stronger backbone can result more accurate selection of augmentations in GeNIe, thereby enhancing the final accuracy. To clarify, DeiT-B is utilized solely as fo(.) for sampling angmentations and as the classifier. Therefore, the observed improvement is attributed exclusively to better augmentation sampling.
1owNee111
RaeNec-50
Angressation
Notue Ratio Solector
Method
I-shoo
S-shot
Angmennaation
Nolen Selector Backhhon fel-)
Method
Teshot
S-hhot
Rackhome fe()
[llnslfeer Backhomn)
[Chestfier Barkhoone)
Oatiee (Oun)
UniSam RanNec11
75.520.6
$5.420.44
Gollee Golle-AAda
UniSum ReaNetio
77.320.6
87.2:0.4
Oelle-Adk
MusiinBosNaai IN [X-I((DXT--)
UniSion ReNNe118
76.820.66
55.020.4
UriSiam Rosdie50 DN DX-I8/De 3)
UniStani ReaNett
78.620.6
87.0:0.4
UniSion ReNNe11
77.520.5
56.3:0.2
Golle-Ada
UniStan ReaNetto
79.2:0.44
88.3:0.5
A.8 Impact of GeNle with Fine- Tuning:
For all our experiments regarding GeNIe and GeNIe-Ada, we assume that the base diffusion model is aware/has been trained on some samples of the target class. This facilitates the addition of the target class (input as text-prompt) into the generated augmentation, while retaining the low-level features of the source image through partial noising. However, there can be scenario where the base diffusion model does not understand the contradictory text prompt and thus fails to incorporate it into the generated image. As a solution, we can use textual inversion (Gal et al., 2022b) to fine-tune the diffusion model on few images belonging the unknown target class to learn the corresponding embeddings for the target categories. This fine-tuning allows us to learn embeddings specific to the target class, enabling the generation of the desired hard-negative examples. To empirically demonstrate the robustness of GeNIe on these scenarios, we present few-shot classification results mini-Imagenet using GeNIe hard-negative angmentations in Tab. A7, gen- erated by textual-inversion fine-tuning the diffusion model on images of the target class. Note that once the diffusion model is fine-tuned, the procedure to generate hard-nogatives using partial noising and a contradic- tory text-prompl remains the same. As can be seen in Tab. A7, GeNIe+TxlIns performs significantly better than DAFusion baseline, It is important to note that, in this case, we do not utilize any information about the target category labels. DAFusion also employs inctualiiverrioo--based fine-tuning: however, does so without generating herd-iegative samples. This indicates that GeNIe effective even in scenario where the diffusion model is unaware of the target-class.
24

---

Published in Transactions on Machine Learning Research (04/2025)
Table A7: mini- Sintt-ImaeNNet:: We augmentations (5-way, 1-shot) and (5-way, 5-shot) few-shot settings of mini-Imagenet dataset with 2 dilferrntt backbones (ResNet-18 and 50), by using Texlual-Inversion (Gal et al., 2022b) on the aareetclaasses The number of generated images per class is 4 for 1-shot and 20 for 5-shot settings.
Re
RewNet-50
Augmentation
Method
Pro-training
I-shot
S-ahid
Angmentation
Method
Pro-training
I-shot
5-hhot
DAPusion 2024
UaiSiam 2022
misp..
613218
81.0114
DAPasoo 2024
UniSiom 2022
mmup.
85.7211
80.921.22
UniSium 2002
moup.
75.9-0.8
84,0+0.9
Gw Toilly
UniSium 2022
76,2+1.2
86.2+0.9
A.9 Details of Fine-graines Few-shot Classification
Here we provide details of Fino-grained Few-shot Classification experiments.
Datasets: We assess our method on several datasets: Food101 (Bossard et al., 2014) with 101 classes of foods, CUB000 (Wah al., 2011) with 200 bird species classes, Cars196 (Krause et al., 2013) with 196 car model classes, and FGVC-Aircraft (Maji et al., 2013) with 41 aircraft manufacturer classes. We provide detailed information around fine-grained datasets in Table A8. The reported metric is the average Top-1 accuracy over 100 episodes. Each episode involves sampling 20 classes and 1-shot from the training set, with the final model evaluated on the respective test set.
Implementation Details: We enhance the basic prompt by incorporating the superclass name for the fine- grained dataset: "A photo of a <target class>,, a type of <superclass>'. For instance, in the food dataset and the burger class, our prompt reads: "A photo of burger, type of food." No additional augmentation is used for generative methods in this context. We generate 19 samples for both cases of our method and also the bascline with weak augmentation.
Table A8: Train and test split details of the fine-grained datasets, We use the provided train set for few-shot task generation, and the provided test sets for our evaluation. Aircraft dataset uses the manufacturer hierarchy.
Dataset
Classes
Thain sumples
Test
samples
CUB200 (Wah et al, 2011)
200
5994
6794
Food101 (Bossard sl. 2014)
101
75750
25250
Cars (Krause al,, 2013)
8144
8041
Aircraft (Maji et al, 2013)
41
6,667
3333
A.10 Few-shot Classification with ResNet-34 on tieredlmagenet
Table A9: tiered-ImageeNt: Accuracies (% + std) for 5-way, 1-shot and 5-way, 5-shot classification settings the test-set. We compare against various SOTA supervised and unsupervised few-shot classification baselines as well as other augmentation methods, with UniSiam (Lu et al., 2022) pre-trained ResNet-34 backbone.
ResNet-34
Augmentation
Method
Pre-training
1-shot
5-shot
Weak
MAMI, + dist (Flnn et al., 2017)
sup.
51.7+ 1.8 5.20+112
70.3+1.7
Weak
ProtoNet (Snell et al., 2017)
sup.
72.1
Weak
UniSiam + dist (Lu et al., 2022)
unsup.
68.7+0044
85.7+0.3
Weak
UniSiam (Lu ct al., 2022)
unsup.
65.0.10.7
82.510.5
Strong
UniSiam (Lu al., 2022)
unsup.
64.7
82.410.5
CutMix (Yun et al., 2019a)
UniSiam (Ta et al., 2022)
mnsup.
63.8+007
80.3+0.6
MixUp (Zhang al., 2018)
UniSiam (Lu et al., 2022)
unsup.
64.1++0.7
80.0+0066
Ing2Ing"(Land) (Luzi al,, 2022)
UniSiam (Lu et al., 2022)
unsup.
66.120.7
83.140.5
Img2Ing" 4 (Luzi et al., 2022)
UniSium (Lu et al., 2022)
unsup.
70.4.10.7
84.7.10.5
Txt2Ing(IIe (He et al., 20226)
UniSium (Lu ct al., 2022]
unsup.
75.0.10.6
85.410.4
DAFusion (Trabucco et al., 2024)
UniSiam (Lu et al., 2022)
unsup. 64.1+2.11
82.8+14
GeliTe (Ours)
UniSlam (Lu et al., 2022)
unsup. 75.7+0.66
86.0+0.4
GeNIe-AAa (Ours)
UniSiam (Lu et al., 2022)
unsup.
76.910.6
86.310.2
We follow the same evaluation protocol here as mentioned in section 4.1. As summarized in Table A9, GeNIe and GeNIe-AGe Ada outperfor all other data augmentation techniques.
25

---

Published in Transactions on Machine Learning Research (04/2025)
Table A10: Few-shot classification comparison of GeNle-Ada with Txt2Ing minilmagenet.
Method
ResNet-18
ResNet-3d
ResNet-50
1-shot
S-shot
1-shot
5-shot
1-shot
5-shot
Txt2Ing GellIe-Ada
76.92100 77.710.8
06.520.9
77.120.8 78.310.9
86,741.0 87.810.9
77.241.3 79.11111
86.8
47.11.
88.411.2
Nokoa Rato
Nokt Rodor 0.6
No Rotlo
Noloa Roto (-10
Prompt"
Reverse Diffusion
Scurce
Peorpt "A ablid.
Reversie Diffusion
dog?
Reverse
GeNIe Figure A4: Key components of GeNIe: (i) careful choice and (ii) contradictory prompt are key idea
behind GeNIe
All Trajectories
Ratio Of The Nabe
images
Target Centroid
0.0
0.4
0.6
0.8
1.0
Source imae
Figure A5: Analyzing the semantic trajectory of GeNIe augmentations 10 different images of class Mushroom (source image) to class Volcano (larget class).
A.11 Additional details of Long- Tail experiments
We present a comprehensive version of Table to benchmark the performance with different backbone architectures (e.g., ResNet50) and to compare against previous long-tail baselines; this is detailed in Table A11.
Implementation Details of LVIT: We download the pre-trained VIT-B of LVIT (Xu al., 2023) and finetune it with Bal-BCE loss proposed therein the augmented datasct. Training takes 2 hours four NVIDIA RTX 3090 GPUs. We use the same hyperparameters as in (Xu et al., 2023) for finetuning: 100 epochs, Ir = 0.008, batch size of 1024, CutMix and MixUp for the data augmentation.
Implementation Details VL-LTR: We use the official code of VL-LTR (Tian et al., 2022) for our experiments. We use pre-trained CLIP ResNet-50 backbone. We followed the hyperparameters reported in VL-LTR (Tian al., 2022). We augment only "Few" category and train the backbone with the VL-LTR, (Tian et al., 2022) method. Training takes 4 hours on NVIDIA RTX 3090 GPUs.
26

---

Published in Transactions on Machine Learning Research (04/2025)
A.12 Extra Computation of GeNIe-Ada
Given that GeNle-Ada searches for the best hard-negative between multiple noise-ratios r's, it naturally requires higher compute budget than txt2Img that only uses r 1. For this experiment, Ada with r E (0.6, 0.7, 0.8) to compare with Txt2Img. Based this, we only have 3 paths, with steps of 0.1); and for each of which we go through partial reverse diffusion process. E.g. for r 0.6 we do 30 steps instead of standard 50 steps of Stable Diffusion. This practically breaks down the total run-time of GeNIe-Ada to approximately 2 times that the standard reverse diffusion (GeNle-Ade Adaa: total r = 0.6 + 0.7 + 0.8 = 2.1 vs Txt22mg total 1). Thus, be fair, we generate twice many Txt2Img augmentations as compared GeNle-Ada to keep a constant compule budget
use GeNIe-
the methods, following, your suggestion. The results are shown in Table A10. As can be seen, even in this new setting, GeNle- Ada offers performance improvement of 0.8% to 1.9% across different backbones.
Table All: Long-Tailed ImageNettLL:: We compare different augmentation methods on ImageNet-Lil and report Top-1 accuracy for "Few", "Medium", and "Many" sets. indicates results with ResNeXt50. *: indicates training with 384 resolution so not directly comparable with other methods with 224 resolution. On the "Few" set and LiVT method, our augmentations improve the accuracy by 11.7 points compared LiVT original augmentation and 4.4 points compared Txt2Img.
ResNet-50
Method
Many
Med.
Few
Overall Ace
CE (Cui al., 2019)
64.0
33.8
41.6
LDAM (Cao et al., 2019)
60.4
46.9
30.7
49.8
c-TT (Kang et al., 2020)
61.8
46.2
27.3
49.6
T-Norm (Kang et al., 2020)
59.1
46.9
30.7
49.4
Cansal (Tang et al., 2020)
62.7
48.8
31.6
51.8
Logit Adj. (Menon al., 2021)
61.1
47.5
27.6
50.1
RIDE(4E)f (Wang al., 2021)
68.3
53.5
56.8
MiSLAS (Zhong et al., 2021)
62.9
50.7
34.3
52.7
DisAlign (Zhang et al., 2021a)
61.3
52.2
31.4
52.9
ACE} (Cai et al., 2021)
71.7
54.6
23.5
56.6
PaCof (Cui et al., 2021a)
68.0
56.4
37.2
58.2
TADEł (Zhang al., 2021b)
66.5
57.0
43.5
58.8
TSC (Li et al., 2022))
63.5
49.7
GCL (Li ct al., 2022c]
63.0
52.7
37.1
54.5
TLC (Li et al., 2022a)
68.9
55.7
40.8
55.1
BCLF (Zhu et al., 2022)
67.6
54.6
36.6
57.2
NCL (Li al., 2022c)
67.3
55.4
57.7
SAFA (Hong al., 2022)
63.8
53.1
DOC (Wang et al, 2022)
65.1
52.8
55.0
DLSA (Xu et al., 2022)
67.8
54.5
38.8
57.5
ResLT (Cui et al., 2022)
63.3
53.3
40.3
55.1
PaCo (Oui et al., 2021b)
68.2
58.7
41.0
60.0
LWS (Kang al., 2019)
62.2
48.6
51.5
Zero-shot CLIP (Radford et al, 2021)
60.8
59.3
DRO-IT (Samuel & Chechik, 2021)
64.0
49.8
33.1
53.5
VL-LTR (Tian et al., 2022)
77.8
67.0
50.8
70.1
Cap2Aug (Roy et al., 2023)
78.5
67.7
51.9
70.9
GeNIe-Ada
79.2
64.6
59.5
71.5
VIT-B
LiVI* (Xu et al., 2023)
76.4
59.7
42.7
63.8
VIT (Dosovitskly al., 2021)
30.5
23.5
31.6
MAE (He et al., 2022a)
74.7
48.2
54.5
DefT (Touvrou cl al, 2022)
70.4
40.9
12.8
48.4
LiVT (Xu et al., 2023)
73.6
56.4
41.0
60.9
LiVT + Img2Ing
74.3
56.4
34.3
60.5
LiVT "m
73.8
56.4
45.3
61.6
LiVT + Txt2Ing
74.9
55.6
48.3
62.2
LiVT + GeNIe (r=0.8)
74.5
56.7
50.9
62.8
LiVT + GeliIe-Ada
74.0
56.9
52.7
63.1
27

---

Published in Transactions on Machine Learning Research (04/2025)
A.13 Further analysis of semantic shifts using GeNIe
In Fig. 5, empirically demonstrate that by increasing the noise ratio from 0 to 1, the semantic calegory of the source image transitions gradually from the source class to the text-prompt's target class. To establish this further, we now choose 10 samples of a source class of Mushroom and generate GeNIe angmentations with the target class of a Volcano. The generated images corresponding to each r E [0, 1] are passed through a DINOv2 encoder and their embeddings are projected onto their 2 principle eigen vectors using PCA. The trajectories extracted from each of these 10 source images is depicted collectively and individually in Fig. A5, It can be noticed that each of the trajectories demonstrate gradual transition of semantic category from the source to the target class, with a sparse distribution of points usually observed within [0.4, 0.6]. This is also observed in the plot on the bottom-right side of the figure where all trajectories are collectively plotted. Here, however, there is no clear range of r where a sparse distribution of points can be observed, thus indicating that each source image has its own optimal r value. This can be attributed to the inter-sample variances of images belonging to the same class. Since GeNIe-Ada operates on each individual source image and target class text-prompt, it facilitates the selection of the best hard-negative per sample.
A.14 How does GeNIe control which features are retained or changed?
We instruct the diffusion model to generate an image by combining the latent noise of the source image with the textual prompt of the target category. This combination controlled by the amount of added noise and the number of reverse diffusion iterations. This approach aims to produce image that aligns closely with the semantics of the target category while preserving the background and features from the source image that are unrelated to the larget.
To demonstrate this, in Figure A4, We are progessivley moving towards the two key components of GeNle: (i) careful choice of r and (ii) contradictory prompt. The input image is a bird in a cage. The top row shows a Stable Diffusion model, unprompted. As can be seen, such a model can generate anything (irrespective of the input image) with large r. Now prompting the same model with "a photo of bird" allows the model to preserve low-level and contextual features of the input image (up to = 0.7 and 0.8), until for a large 2 0.9 it returns bird but the context has nothing to do with the source input. This illustrates how careful choice of can help preserve such low-level features, and is a key idea behind GeNTe. However, we also need a semantic switch to a different target class as shown in the last row where a hardly seen image of a dog in a cage is generated by a combination of a careful choice of r and the contradictory prompt leading to the full mechanics of GeNle. This sample now serves as hard negative for the source image (bird class).
A.15 Analyzing Noise Effects in Bi-Directional Transformations with GeNIe
To further explore the effect of noise ratio r in GeNIe, we conducted an experiment where GeNIe was applied twice to transform between a source image and a target category. For this experiment, images from the "mushroom" category were used as the source, and "volcano" served as the target calegory. In the first step, we applied GeNIe using mushroom image as the source and a volcano prompt as the target. In the second step, we reversed the process: the GeNIe-generated volcano image from the first step was used the source, with the target prompl set to mushroom. Importantly, using a smaller noise ratio, during the generalion of the volcano image helps preserve more low-level visual features from the original mushroom source image, Consequently, when the roles of source and target are flipped in the second step, the final image retains a stronger resemblance to the original mushroom source image for lower noise ratios. This phenomenon is visualized in Fig. A6. As shown, a lower noise ratio during the first step results in the preservation of more visual features, leading to a final image that more closely resembles the original mushroom source.
A.16 More Visualizations
Additional qualitative results resembling the style presented in Fig. 4 are presented in Fig. A8, and more visuals akin to Fig. 2 can be found in Fig. A9. Moreover, we also present more visualization similar to the style in Fig. in Fig. A7.
28

---

Published in Transactions on Machine Learning Research (04/2025)
Prompt: photo of Volccno,
Prompt photo Mushroom.
Mushroom Valcano
Scare taage
Mushroom Generated Volcano Mushroom
Target Centroid
Source (10 samples)
Prompt
Prompt: A photo of Mushroom.
Prompt A photo of Volcana.
Promot A photo of Volcano.
Prompt: A photo Mushroom..
Seave
1+14
-13
Figure A6: Trajectory of GeNIe augmentations: further analyze the effect of noise ratio r GeNIe, we conducted experiment using a of augmentations generated from 10 different, source images in the 'mushroom' calegory, with target label of "Volcano," were processed through the DinoV2 ViT-G model, which serves our oracle, to extract their embeddings. For visualization, applied PCA to these embeddings. Next, we selected one augmentation with specific noise ratio, (r), and used it the source image in for the 'volcano' category in GeNIe, with the target prompt set to 'mushroom. As observed, using lower noise ratio samples as the source for "volcano" preserves more low-level visual features from the original mushroom source image. Consequently, after a second round of applying GeNIe, the resulting augmentations (even rows) tend to more closely resemble the original source image (first image in the corresponding odd rows above). The plot presents the embeddings visualization sample, showcasing the impact of varying noise ratios used in the second step of applying GeNIe.
varying noise ratios. Similar to Fig. 5, all generated augmentations
10 samples, while right plot provides a detailed
29

---

Published in Transactions on Machine Learning Research (04/2025)
Canerited inge
0.0 022
0.6
0.5
68
10
Mancelnaje
Vaurre Elano close hore
Seare buget
Promgt A photo of leeoriam.
Source Eaget
Soares Iaage
mi33
n:5
1667
1100
Doper Exorsso
Promot: A photo of Mustroom
Sourc Clas: ab) Canc Mashoen
Image
vnyy
Source Chex Target
Orange
Classe hemagra I Target Cleos Ohange
Scunce Image
H67
MO0
Prompet: A photo Tonano
Prompt p oto of Earder Calio..
Source Dlas Creater dog Trrpet Bender
Scarce Image
MB3
m03
mos
H67
m00
Figure A7: Effect of noise in GeNIe: Similar to Fig. pass all the generated augmentations through the Dino ViT-G model, which acts our oracle model, to obtain their associated embeddings.. Subsequently, we employ PCA for visualization purposes. The visualization reveals that the magnitude of semantic transformations is contingent upon both the source image and the specified target category.
30

---

Published in Transactions on Machine Learning Research (04/2025)
Target Image Volcano
Ing2Ing
2 Ing
Txt 2 1mg
Source image Mushroom
GeNIe (r:0.6) GeNIe (r-3.7)
GeNIe(r-D.)
Snoek
German Shophard
Sibcrian Husky
chocolate
Mortar
Red Wine
Face Powder
Ice cream
Potple
Beer Bottle
zuecamole
burrito
karream
anana
Figure A8: Visualization of Generative Samples: More visualization akin to Fig. 4. We compare GeNIe with two baselines: Img2Img' augmentation uses both image and text prompl. from the same category, resulting in less challenging examples. Txt2Img augmentation generates images based solely from the task's 's visual domain. GeNIe augmentation incorporates the target category name in the text prompt along with the source image, producing desired images with an oplimal amount of noise, and balancing the impact of the source image and text prompt.
text prompt, potentially deviating
31

---

Published in Transactions on Machine Learning Research (04/2025)
Prompt: A plolo of cal.
C
SeMIel
GelTT (mm283)
ColIe
C-N (H075)
Source mags
Prompt: photo
image
GeMIe (HOvE
CcRIa m0y
Gellee (-RUG)
GoRIo P nXt
Colloweigg
Prompt:
Source Image
Gol 1e F
GeN le (-2.0)
GelIIe (n0
GeNIo0
GeNle 1H0.0
Gel: (H0.2))
GeXIeenccr
Prompt: A photo of bird
Sourte
Get
Ge (e0t00
BeNEer--n0
Prompt: setellite photo pond from top
Ge
GeMTe
GeNDe ino
GeNIeovo
GeNIon-KK
Prompt: A sotellte shoto of parking lot from top
Source
RaNTamdd
GeN
GaNIninG
Prompt: sateilno
ColIeek 0000
GeXie (50
GaNTancc mv
GeNIeovam
GeN
Ge
Figure A9: Effect of noise in GelIl:: Akin to Fig. 2, we use GeliIe to create augmentations with varying noise levels. As is illustrated in the examples above, reduced amount of noise leads to images closely mirroring the semantics of the source images, causing a misalignment with the intended target label.
32


Reviews:
[{"review_id": "B4hpZHHAo1", "reviewer": "Reviewer_rDm2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n1. A data augmentation methods for improving downstream classification performance is important and has practical needs.\n\n2. Comprehensive discussion of related work on data augmentation using generative models.\n\n3. Extensive evaluations on few-shot and long-tailed classification tasks.\n\n**Weaknesses**:\n1. Lack of comparison with more recent mixup methods: The paper evaluates GeNIe against mixup and cutmix, which combine images from different categories. However, these methods are outdated, and improved mixup variants have been proposed. For example, saliency-based augmentations were specifically proposed to address the issue of unrealistic augmentations, as they incorporate saliency maps to preserve important features. Such methods should be included as baselines. [1]\n\n2. Limited evaluation: The paper only evaluates few-shot and long-tailed classification, omitting standard benchmarks. The rationale for this choice should be clarified.\n\n3. Ambiguity in the novelty claim: My understanding is that GeNIe’s novelty lies in its noise-level adjustment mechanism. However, the first contribution statement is somewhat ambiguous and should be rephrased for greater clarity and accuracy.\n\n[1] Ma et al., SAGE: Saliency-Guided Mixup with Optimal Rearrangements", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes GeNIe that leverages diffusion models to generate additional images for improved downstream classification tasks. To do so, GeNIe uses a text-conditioned diffusion model to synthesize samples that preserve key features of the source image while incorporating semantics from the target category. To further improve GeNIe, an adaptive approach is proposed to optimize the noise level during denoising. Empirical evaluations on few-shot and long-tailed classification tasks demonstrate GeNIe's efficacy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Questions**:\n\nGeINe is very similar to mixup-based approaches, with the key distinction that it retains one-hot labels. It would be interesting to explore whether GeNIe could incorporate mixup’s strategy for combining category information from two sources. Also, the parameter $r$ being in $(0,1)$ resembles the interpolation factor used in mixup. Have you tried using $r$ instead?\n\n**Other requested changes**: \n\nSee weakness.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3tWJ1SqhO5", "reviewer": "Reviewer_LTne", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n* Overall the paper is very structured and easy to follow. Figures and Pseudocode are used well to illustrate the approach\n* The method investigates a new way of integrating diffusion-based Augmentations for downstream tasks. While most previous methods rely on positive augmentations, to my knowledge, this is the first approach that aims to generate negative ones. \n* The method's effectiveness is studied with extensive experiments and a proper choice of Diffusion-based augmentation baselines. Furthermore, several ablation studies have been conducted to highlight the role and impact of noise levels and explain the method's effectiveness. \n\n**Weaknesses**\n* Since the model is trained on a large labelled dataset, the non-diffusion baseline is not fairly chosen. The use of the diffusion models introduces more class information as is available in the Few-Shot and Long-Tailed Classification", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a method for generating hard negative samples using a latent diffusion model, conditioned on a noisy source image and a target label. These negative samples help to improve the robustness and performance of a logistic regression classifier on the UniSiam backbone for Few-Shot Classification, as well as fine-tuned models for Long-Tailed Classification. Additionally, the authors conduct several ablation studies to analyze the impact of the Diffusion model and the chosen noise level of the source image.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* One major concern is the assumption that a large labelled dataset is available for training the diffusion model (In this case LAION). However, if such a dataset is accessible, a natural and strong baseline would be a classification or CLIP model trained directly on the labelled LAION dataset. The embeddings from this model could then be used to perform Few-Shot Classification and Long-Tailed Classification. To fully support the claims made in the paper, I strongly suggest that the authors conduct this baseline experiment. This comparison would clarify whether the benefits of their approach come primarily from the generated negative samples or from the additional label information obtained by training on LION. Otherwise, the claim \"significantly improves the downstream classification performance.\"  would have to be put into context. \n* In the discussion of \"rapid change of image content based on r\", I believe [1] has to be cited\n* (Minor detail). In the last line of Algorithm 1 it should be r/M instead of r/n I believe?\n\n1. Sclocchi, Antonio, Alessandro Favero, and Matthieu Wyart. \"A phase transition in diffusion models reveals the hierarchical nature of data.\" Proceedings of the National Academy of Sciences 122.1 (2025): e2408799121.", "broader_impact_concerns": "I do not have concerns about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bZ4BSkSvii", "reviewer": "Reviewer_ipeF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:   \n1. Good presentation, well explained and written  \n2. Good experimental results   \n 2a. Moderate gains on few-shot classification compared to several baselines, the closest one being “Txt2Img”   \n 2b. Larger gains on long-tail classification and fine-grained classification   \n3. Useful and interesting analysis experiments and ablations, in particular around the effect of the amount of noise and the automatic noise selection   \n4. Shared code\n\n\nCons:\n1. At the high level, I would like to see a better motivation of why the experimental settings evaluated in the paper are practically relevant. In particular, whether the proposed method still offers improvements given existing VLMs that would likely solve the tasks well out of the box. There’s a CLIP baseline in one of the experiments, but CLIP is by far not the newest model. \n2. A couple of questions about the experimental settings:  \n2a. In the few-shot experiments, there’s a note “Note that this corresponds to an inference-only FSL setting where a pretraining stage on an abundant dataset is discarded”. Does it mean the models are trained from scratch? Why, why is this setting relevant when pre-trained models are available? What does “unsup” in Table 1 mean, then?  \n2b. What is the motivation for the setting used in fine-grained classification, with an SVM? Is it a standard setting used in previous work? Why not fine-tune instead of SVM?  \n3. Speaking of the results, in few-shot experiments, gains over Txt2Img baseline are not large, especially in the compute-controlled setting (Table A7). It’s great that these experiments have been performed, though  \n4. there’s a phrase in the introduction “Combating such a bias or lack of sufficient data samples during model training is essential in building robust models for self-driving cars or surveillance systems, to name a few” - if this is the motivation, would be useful to have experiments go more into this direction too. On a related topic, mentioning surveillance as one of the key applications may raise some ethical concerns   \n5. Small presentation comment: the introduction starts with “Augmentation has become…” - it should rather be “Data augmentation”.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method for data augmentation using a text-to-image diffusion model. Specifically, in the image classification setting, the idea is to modify real images by adding some noise to them and denoising them with a diffusion model conditioned on prompts corresponding to other classes. This results in images that are in some respects similar to the original image (e.g. the overall colors/layout/background), but feature an object of the “new” class the prompt specifies and as such belong to this new class. \n\nSince the right desired amount of noise to change the class but keep some properties of the original image is image/prompt-specific, the paper also proposes a method to automatically determine a suitable amount of noise, by looking at the behavior of features extracted from the image and automatically detecting when they switch from being closer to the “original” to “new” class.\n\nThe method is evaluated on ImageNet-derived tasks of few-shot classification and long-tail classification, as well as on fine-grained classification on several other datasets. The method performs well compared to relevant baselines. The paper also presents several interesting and relevant analyses and ablation experiments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Answer questions and address the concerns in \"cons\" above", "broader_impact_concerns": "Generally no specific concerns - it's a general method for data augmentation. The paper has a reasonable \"Broader Impact\" subsection. One point I am somewhat concerned about is the mention of surveillance as one of two key potential applications - I do not see why this work would be suitable specifically for surveillance and therefore would not call it out.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel augmentation strategy using diffusion models to create hard negative samples for classification tasks. The approach is simple yet effective and is extended via GeNIe-Ada, which adaptively selects the noise level per sample to optimize the quality of the generated augmentations.

**Reviewer Summary and Discussion:**

All three reviewers leaned toward acceptance after the author rebuttal:

- **Reviewer rDm2** appreciated the added SAGE baseline and the clarification on novelty, shifting from a leaning reject to a leaning accept.
- **Reviewer LTne** highlighted the novelty of using diffusion models to generate *negative* augmentations and found the method well-illustrated and experimentally sound.
- **Reviewer ipeF** found the paper well-written and the experiments interesting, but noted that the practical relevance relative to large-scale VLMs like CLIP could be better clarified. These concerns were partly alleviated by new CLIP-based baseline experiments (Appendix A.5, Table A3), where the authors evaluated GeNIe and GeNIe-Ada using a CLIP-pretrained ResNet-50 and showed measurable improvements (e.g., +1.3% in 1-shot accuracy) over strong CLIP baselines.

Overall, the author responses were thorough, constructive, and addressed the raised concerns. The method demonstrates consistent improvements across diverse setups, and the analysis is both detailed and well-supported.

**Requested Minor Revisions:**

- Clarify the positioning of the method relative to large-scale vision-language models (e.g., CLIP), particularly regarding performance tradeoffs and applicability in resource-constrained settings.
- Clarify novelty positioning: while GeNIe introduces a relatively underexplored angle (generating hard negatives using diffusion), the paper could benefit from a clearer articulation of how this differs from existing diffusion-based augmentation approaches, while clearly distinguishing it from related diffusion-based methods.
Decision: Accept (poster)