Data Augmentation with Diffusion for Open-Set Semi-Supervised Learning
Authors: Seonghyun Ban
Heesan Kong
Kee-Eung Kim
Venue: NeurIPS 2024 poster (2024)
Keywords: Semi-supervised learning
Class distribution mismatch
Diffusion model.
Primary area: machine_vision

Abstract: Semi-supervised learning (SSL) seeks to utilize unlabeled data to overcome the limited amount of labeled data and improve model performance. However, many SSL methods typically struggle in real-world scenarios, particularly when there is a large number of irrelevant instances in the unlabeled data that do not belong to any class in the labeled data. Previous approaches often downweight instances from irrelevant classes to mitigate the negative impact of class distribution mismatch on model training. However, by discarding irrelevant instances, they may result in the loss of valuable information such as invariance, regularity, and diversity within the data. In this paper, we propose a data-centric generative augmentation approach that leverages a diffusion model to enrich labeled data using both labeled and unlabeled samples. A key challenge is extracting the diversity inherent in the unlabeled data while mitigating the generation of samples irrelevant to the labeled data. To tackle this issue, we combine diffusion model training with a discriminator that identifies and reduces the impact of irrelevant instances. We also demonstrate that such a trained diffusion model can even convert an irrelevant instance into a relevant one, yielding highly effective synthetic data for training. Through a comprehensive suite of experiments, we show that our data augmentation approach significantly enhances the performance of SSL methods, especially in the presence of class distribution mismatch.

Data Augmentation with Diffusion for Open-Set Semi-Supervised Learning
Seonghyun Ban Heesan Kong",, Kee-Eung Kim 4,2
Kim Jaechul Graduate School of AI, KAIST
School of Computing, KAIST
shbangai. kaist ac ac.kr, kr, hskongßai. kaist. ac.kr, kekim@kaist.ac,
Abstract
suneriuper learning (SSL) seeks utilize unlabeled data to overcome the limited amount of labeled data and improve model performance. However, many SSL methods typically struggle real-world scenarios, particularly when there a large number of irrelevant instances in the unlabeled data that do not belong to any class the labeled data. Previous approaches often downweight instances from irrelevant classes to mitigate the negative impact of class distribution mismatch on model training. However, by discarding irrelevant instances, they may result in the loss of valuable information such invariance, regularity, and diversity within the data. this paper, we propose a data-centric generative augmentation approach that leverages diffusion model to enrich labeled data using both labeled and unlabeled samples. A key challenge is extracting the diversity inherent in the unlabeled data while mitigating the generation of samples irrelevant to the labeled data. To tackle this issue, we combine diffusion model training with discriminator that identifies and reduces the impact of irrelevant instances. We also demonstrate that such a trained diffusion model can even convert an irrelevant instance into relevant one, yielding highly effective synthetic data for training. Through comprehensive suite of experiments, we show that our data augmentation approach significantly enhances the performance of SSL methods, especially in the presence of class distribution mismatch.
Introduction
Deep neural networks (DNNs), trained using a large amount labeled datasets, have shown to achieve remarkable performance in variety of supervised learning tasks, such as image classification (LeCun
et al. 2013 Krizhevsky et al.) 2017) and object detection (Everingham et al.] 20100; Lin et al.
2014). Nonetheless, the intensive labor of annotating vast datasets often renders the construction of
sufficiently large labeled datasets prohibitively expensive for numerous applications (Oliver et al.
2018). To address this, semi-supervised learning (SSL) (Chapelle et al. 2009) has emerged viable approach, which aims to leverage abundant unlabeled data to overcome the limited availability of labeled data.
Recent progress in SSL has made many noteworthy advancements, including techniques such as pseudo-labeling (Lee) 2013 Pham al. (021). consistency regularization (Sajjadi et al. 2016
Tarvainen Valpola 2017 Sohn et al. (020), and entropy minimization (Grandvalet & Bengio
Miyato et al. 2018 Berthelot et al.] 2019). However, common limitation of these methods is their reliance on the critical assumption that both unlabeled and labeled data instances are drawn
"Equal Contribution.
Conrespondigg Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Unlabeled
Noised images
Transformed images
Labeled images
images
Truck
(Tree)
(Tree)
Truck
Car
(Fish)
(Fish)
Ship
(Mountain)
(Mountain)
Ship
Ship
Forward process to time step T
process condition "XXX" to time step 1-1, 0.
XXX
Figure I:
Transforming unlabeled data using a diffusion model. Initially, the unlabeled data includes classes like trees, fish, and mountains, which irrelevant to the labeled data's classes such as trucks, cars, and ships. The reverse process with class conditioning resolves this mismatch while preserving the diversity of the original unlabeled samples. More examples be found in Appendix H
from an identical distribution. This often leads to significant performance degradation when there is a class distribution mismatch, as noted in Oliver et al. (018). Given that real-world settings often deviate from this assumption (Guo al. 2020). it becomes crucial to address the class distribution mismatch for successfully applying SSL in realistic scenarios.
A prevalent approach to mitigate the class distribution mismatch in SSL involves selectively utilizing unlabeled data through a weighting function which serves to reduce the influence of irrelevant unlabeled samples (Chen et al. 2020 Guo et al., 2020). While these filtering methods are intuitively appealing and have shown their effectivencss mitigating negative effect of the class distribution mismatch, they may result in the loss of valuable information such invariance, regularity, and diversity within the data: even though the labeled data contains only white trucks, could we generate images of red trucks using the red bike images in the unlabeled dataset?
In this paper, we propose generative data augmentation approach that leverages diffusion model to enrich labeled data using both labeled and unlabeled samples. A key challenge lies in utilizing the diversity of the unlabeled data compensate the limited amount of labeled data, while minimizing the generation of samples that are irrelevant to the classes in the labeled dataset. To address this, we integrate diffusion model training with a discriminator that evaluates the relevance of each unlabeled instance. The discriminator's resulting score is used to assign weights unlabeled instances, allowing those with higher relevance to contribute more significantly to the training of the diffusion model.
In addition, drawing inspiration from the approach in Meng et al. (022),, we add noise to each unlabeled sample and utilize them as guide images during the data generation process. As depicted in Figure [ we found that incorporating class conditions into this generation process can transform possibly irrelevant unlabeled samples into labeled samples while preserving key characteristics the original unlabeled samples (e.g., outline, color arrangement, shape, etc.).
Our extensive experimental results, utilizing CIFAR-10. CIFAR-100 (Krizhevsky & Hinton 2009). ImageNet-30 (Deng et al. 2009), and ImageNet-100 100 (Cao et al. 2022) datasets with six baseline methods, demonstrate that our approach further improves the performance of recent SSL methods, especially under the class distribution mismatch.
Related Works
2.1 Standard Semi-Supervised Learning
Semi- Siii-siupeeieed Learning (SSL) aims to leverage both labeled and unlabeled data mitigate issues related the scareity and high annotation cost of labeled data. Since many SSL methods operate under the assumption that labeled and unlabeled data are sampled from an identical distribution,

---

we refer to this as the standard SSL setting. Among the various approaches to the standard SSL setting, we briefly review two of the most representative methods: pseudo-labeling and consistency regularization.
Pseudo-laheling (Scudder T965
2013 is a technique where the model- predicted labels of the unlabeled data are treated as if they were true labels. Essentially, this method simply converts unlabeled data into labeled data. On the other hand, consistency regularization (Bach- man et al. 2014) has become a crucial component in recent SSL regimes. This approach applies data augmentation on the unlabeled data to regularize the model to yield similar outputs for augmented views of the same instance (Sajjadi et al. 2016 Laine & Aila 2017 Sohn et al. 2020).
McLachhann 1975 Lee
However, these methods often fail when there is a mismatch between the distributions of labeled and unlabeled data. In some cases, their performance may be even worse than simply discarding all the unlabeled data (Oliver et al.] 2018). The primary source of the problem is the presence of unlaheled instances that do not belong to any of the classes present in the labeled data, which we refer to as out-of-distribution (OOD) instances. They may exacerbate confirmation bias (Arazo et al. 2020) in pseudo-labeling approaches or intensify the overconfidence problem in consistency regularization approaches (Chen al. 2020
2.2 Open-set Semi-Supervised Learning
The open-set SSL setting refers to the practical yet challenging scenario where the class distributions of labeled and unlabeled dataset differ significantly. A prevalent solution to the class distribution mismatch is to filter out OOD instances from the unlabeled dataset. To achieve this, it is necessary to accurately identify them, despite the absence of label information. Uncertainty-Aree -waree Self- Distillation (UASD), proposed by (Chen et al.] (2020), formulates temporally ensembled networks and utilizes ensemble prediction to quantify predictive uncertainty of labels to identify OOD instances. DS3L (Guo et al. 2020) adopts a meta-learning approach to selectively use unlabeled data that enhances generalization performance. OpenMatch (Saito et al. (02T) leverages one- onelvs-all classifiers as the OOD detector to filter out OOD instances. Safe-Student (He et al. 2022) employs teacher- student mechanism and introduces energy- peergy-discrepaccy, a new scoring function for detecting OOD instances. The above methods follow the detect-and-filbe paradigm, which is the dominant approach of open-set SSL, assuming that OOD instances are fundamentally harmful.
In contrast to these detect-and-filee approaches, several studies share similar goals to ours, aiming to harness the potential of OOD unlabeled data rather than simply discarding them. T2T (Huang et al., 2021) incorporates warm-up training step using OOD instances to perform self-supervised pretext task for learning effective discriminative features. TOOR (Huang et al.) 2022) introduces a weighting mechanism to evaluate the transferability of each OOD instance based on domain similarity and class tendency, and uses adversarial domain adaptation to align the feature distributions of transferable OOD instances and in-distribution (ID) instances. Fix-A-Step (Huang et al. 2023) leverages OOD instances to obtain useful data augmentation to promote diversity of training data, and integrate this idea into MixMatch (Berthelot et al., 2019). IOMatch (Li et al. 2023) takes into consideration that the OOD detector may be unreliable, particularly when labeled data are scarce. instead employs multi-binary binary classifier to produce unified open-set pseudo-labels for labeled and unlabeled data, including OOD instances.
Our approach significantly deviates from these open-set SSL methods. It is primarily centered on developing an effective data augmentation strategy from both labeled and unlabeled data, which can be used to transform an unlabeled OOD instance into labeled ID instance.
2.3 Recent Diffusion- based Augmentation Approaches
In this section. we provide a detailed comparison of DWD with diffusion iiffusion-based methods, namely DPT YYou et al. )023) and DA-Fusior! Trabucco al. 2024
DPT is a simple yet effective method that integrates diffusion models into SSL. To generate semanti- cally accurate images, they trained a semi-supervised classifier on partially labeled real images and used to assign pseudo-labels for all the data. Subsequently, they trained conditional diffusion
Although DA DA-Fusion is not a SSL algorithm, we include it as a supervised learning baseline due to its methodological similarity in utilizing image-to image-to-image to-image generation process.
3

---

model using the pseudo-labels to synthesize images and then re- re-trained the semi- supervise classifier using these generated images. However, they still rely on the fundamental assumption behind the standard SSL. setting. When faced with mismatch in class distributions, they suffer from the con- firmation bias in pseudo labels of OOD unlaheled images and become ineffective. In contrast, our approach addresses the class distribution mismatch by integrating the discriminator into the training of the diffuuson model.
DA-Fusion is a data augmentation method that utilizes a large pretrained text-to-image diffusion model (i.e., Stable Diffusion, Rombach et al. 2022). Similar to ours, also initiates the reverse process with partially noised real images rather than generating images from scratch. However, the purpose of the generation process is distinctly different from that of our approach. While DA-Fusion aims to augment given labeled samples with subtle visual details already contained in the pretrained diffusion model, our method properly trains diffusion model to capture both the labeled data distribution and the diversity of unlabeled samples from the given datasets, and transforms irrelevant unlabeled samples into labeled ones.
3 Preliminaries
Diffusion models (Sohl-Dickstein et al., 2015) incrementally add Gaussian noises to the data during its forward process and progressively removes this noise during the reverse process to reconstruct the original data. Given a data point xo, the forward process is defined as a Markov chain that produces a sequence of noisy samples XI...., XT according to a variance schedule 32,... Br:
T
((x, x-) = N(x: V 1,
((xrr) Xo) = II (xxx)x
(1)
The forward process exhibits a notable property in that at any arbitrary time step t E {l, T} can be obtained in closed form:
X V + v(1 (t)e
where og = 3r, a, IT= 02 and I). As the reverse process can be expressed using the same functional form when 3, is sufficiently small (Feller) [949] Sohl-Dickstein et al.) (013)).
the reverse process is also defined as a Markov chain with learned Gaussian transitions starting p(xr) = -N(x. I):
T
po(x; xx) == M(x--1;00 1), Ex (xxx)))); ()),
po(xo:)) = p(xT) II po (x,-1
(3)
Denoising Diffusion Probabilistic Models (DDPMs) Ho et al. (2020) propose
simplified
Gaussian distribution parameterization of the reverse process, which sets Eg to time-dependent constants and reparameterizes the mean function approximator μa (x,,,) t) with noise predictor cx t)
which predicts e in 2 The reverse process then learned by the following objective:
Ladpm Exx.,.e (x)oxx t) <l
For the conditional DDPM, the primary modification is the incorporation of conditions (such
classes or texts) additional input, expressed (x c, t) This adaptation allows the diffusion model to take into account specific conditions or attributes during the reverse process, thereby enhancing its applicability to more targeted scenarios.
Positive-Unlabeledd (PU) Learning (Liu et al. 2002 Li & Liu. 2003; Du Plessis et al. 2015: Kiryo
et al. 2017) is a binary classification task in situation where negative labels are missing. It aims to train models using positive-labelle and unlabeled data to perform binary classification. The main idea involves indirectly estimating the model loss negative samples using the class prior. Given that unlabeled data are drawn from p" (x) = (xp((x) (x) + (1 (4) p (x), where μ = p(Y = 1) is
the class prior and p+ +x) p(x Y 1) and p (x) = p(x Y -1) are positive and negative class-conditional densities, the loss can be reformulated as:
E [[((((x), Y)] = E.EE [[(g(x), 1)] + (1- E,, [[(g(x), 1]] ,IE,, [x(g(x), 1)] μ'E,1 [x(g(x), -1]] + Ep- [((g(x), -1)] Here, E denotes loss function and g represents the model.
(5)

---

Class
Embedding
Labeled data xf
Frog
Ship
Horse
Diffusion Model
DWD loss
Unlabeled data x
LDwD
Discriminator
Weights of x*
Figure 2: Schematic diagram of Discriminator-Weieee Diffusion (DWD). The conditional diffusion
model is trained using both labeled and unlabeled data. The unlabeled data is utilized for unconditional training without class conditions. The pre-trained discriminator assigns weights to each unlabeled data sample to mitigate the potential negative impact OOD samples.
4 Methodology
In this section, we introduce our data augmentation method, which leverages diffusion model to generate synthetic data address the scarcity of labeled data. As previously discussed, we start with training the diffusion model on labeled and unlabeled data while mitigating the class distribution mismatch. Subsequently, we employ the reverse diffusion process to transform unlabeled samples into synthesized labeled samples.
4.1 Training Diffusion Model
A simple training scheme for SSL data The diffusion model trained only on the labeled data will inevitably overfit and merely generate replications due to their limited amount. It is thus essential to train the diffusion model on both labeled and unlabeled data, while taking advantage of the label information. We adopt class-conditional diffusion model, shown in Figure where labeled data are used for conditional training while unlabeled data are used for unconditional training, trained with the loss
Lyemi Eisele) (x,, c, - -e||a] Ex._-,,, E
(6)
where a serves as a hyper-parameter that controls balance between labeled and unlabeled data, Dr and Du represent labeled and unlabeled datasets. Intuitively, this straightforward objective aims to train the diffusion model to reconstruct not only the limited labeled data but also abundant unlabeled data, thereby regularizing the model against overfitting to the labeled data. It is noteworthy that this approach shares similarities with the consistency regularization technique in the sense that the unlabeled data utilized for regularization purposes.
Discriminator However, when we train the diffusion model with the loss in ( OOD unlabeled samples can have negative impact on capturing important characteristics of the labeled data distribution. These samples should be made contribute less towards the overall training loss. In line with aforementioned filtering methods (Chen et al., 20200 Guo et al., (020), we leverage discriminator to weigh the unlabeled data instances. The discriminator is tasked with differentiating between positive samples that are closely aligned with the distribution of labeled data and negative samples that are irrelevant. To train the discriminator, we adopt PU learning with the training loss
Laisc f Ex-D,, [olog (x) log (1 dx + Ex [ lolog (1 (x))),
(7)
where dg denotes the discriminator parameterized by σ, and μl represents the ratio of positive samples among unlabeled samples, i.e. belonging to one of the classes the labeled dataset. This ratio can either be estimated from the dataset (Menon et al. 2015 Jain al. 2016 Christoffel et al. 2016) or treated as a hyperparameter.
We then use the discriminator assign weights the unlabeled instances so that they align with the distribution labeled data. simple algebraic manipulation tells us that the following weight formula yields unbiased loss estimation via importance sampling:
5

---

Algorithm Discriminator- Weightedd Diffusion (DWD) Training
Input Labeled dataset Dj and unlabeled dataset Du.
Parameter Learning rate fe. Te. hyper-parameter μ., a, and total number iterations Kg. Kg.
Either pretrain the diffusion model сœ on Du or acquire an off-the-shelf pretrained diffusion model. # Train the discriminator dæ using objective (7):
for 71 0, 1, 2, K do Sample batch of data x² Dr and x" Du-
Φ + Φ 7o v [u. x [-log dx(x') ) + log (1 dx(x))) Rx- Fx-[--log (1 - (xxx)))
end for
# Finetune the diffusion model Eg using the discriminator de and weighting function w from (
for n 0, 1, 2,.,, Ko do Sample batch of data (x',c) ~ Dr and x" ~ Du- 0 <- 1o Vo [Ext,,elleat c, t) e <l + a Te a.e (w(()) [|Eo(x),)) -=] end for
Output Learned diffusion model 6g and learned discriminator dø.
Proposition 4.1. Given optimal discriminator d', using the following importance weight unlabeled data yields an unbiased estimation of the loss function with respect to the labeled data distribution:
d(x)
w(x)
(x(()) + (1 11 d(x))
For detailed proof, please refer to Appendix A The final training loss for the diffusion model then becomes:
LDWD (xx. (x c, t) < + a Exo (xx_)),)___((())ll))(l(/ () <l
(9)
The overall training scheme, referred as Discriminator- Weighttd Diffusion (DWD), outlined in Algorithm
4.2 Seeding Data Generation with Unlabeled Instances
After training the diffusion model, a straightforward approach to generating synthetic data would be to start the reverse diffusion process from Gaussian noise. In our methodology, however, we begin the reverse process using partially noised image an unlabeled instance. we will show in Section [53 we found that this approach leads to additional performance gains. It successfully transforms an OOD instance into an in-distribution sample while preserving some important characteristics the original sample (see Figure IT. Thus we can exploit the diversity present in the unlabeled data when generating synthetic in-distribution data. The procedure is detailed Algorithm in the Appendix B
We can think of two usage scenarios for the samples generated from the diffusion model: we could take the samples as pseudo-labeled synthetic data to enrich the labeled data and employ fully supervised learning method, or take the samples as a transformed unlabeled data by discarding the class conditions and employ an SSL method. In the latter case, we expect that the SSL method will perform better, since the class distribution mismatch has been mitigated.
5 Experiments
To assess the effectiveness of DWD, we conduct experiments across a broad set of tasks in two settings. (1) DWD-SL: fully supervised learning setting where the unlabeled dataset is converted to a pseudo-labeled dataset by replacing the instances with their transformations along with their class conditions as target labels; and (2) DWD-UT: an SSL setting where the unlabeled dataset is replaced by the transformed unlabeled samples and employ the baseline SSL method.
Tasks The SixAnimal task Oliver et al. (2018) uses CIFAR-10 dataset to classify six animal classes (bird, cat, deer, dog, frog, and horse). Following the setup in Huang et al. (2023). we sampled 400 images per class for the labeled dataset and included up to 4100 images per class in the unlabeled dataset. To investigate the impact of class distribution mismatch, varied the mismatch percentage

---

Table I:
Performance comparison on four tasks. We report the mean accuracy averaged over three seeds, along with standard error. Top scores for each task are highlighted.
Task Name
MixMatch
FixMatch
MPL
OpenMatch
Fix-A-Step
IOMatch
DWD-SL.
SixAnimal (C = 75%)
80.77=0.11
82.5020.16
65.62:0.47
80.3410.21
85.34+0.17
83.05:0.16
85.86:+0.28
CIFAR-10/100
71.02=0.22
78.91
70.95=0.34
70.15+0.30
74.60+0.31
77.6640.22
80.05:00.14
ImageNet-30 30
68.6 08.67=0.37
70.07:0.26
72.65=0.70
72.78:0048
79.67=0.81
79.23:10.29
82.20:10338
ImageNet-100
69.30+0.41
65.11+0.32
68.43++0.3
65.42+0.36
65.80++0.99
66.85+0019
0..81+0.31
Table 2: Performance of standard SSL methods before and after applying DWD-UT. Highlighted scores show significant increases without overlapping intervals.
Mixmatch +DWD-UT
FixMatch +DWD-UT
MPL
Task Name
MixMatch
FixMatch
MPL
+DWD-UT
Sixmmi (( = 75%)
80.77::..11
84.72:10.22
82.50:10.16
87.17=10.19
65.62:1::47
83.88-0.188
Cifar- 10/100
71.02:1:.32
80.47-10.49
78.9110.11
83.80=10.25
70.95:10.34
80.24:0..56
ImageNet-30
68 08.67
85.20-+0.10
70.07+0.26
81.87++0.61
72.65+00.00
90.20++0.23
ImageNet-100 100
69.30:10.41
81.62:10.36
15.112
80.38:10.34
68.43
75.66:10.26
Table 3: Performance of open-set SSL methods before and after applying DWD-UT.
OpenMatch +DWD-UT
Fix-A-Step +DWD-UT
10Match
Task Name
OpenMatch
Fix-A-Step
IOMatch
+DWD-UT
Six (C = 75%)
80.34410.21
85.71-10.3
85,34:10.17
86.68:10.23
83.05:10.16
87.20-10.13
Cifar 10/100
00.15:10.30
80.99:10.03
74.60:00.31
79.02:10.75
77.661:.22
83.22:10.16
ImageNet-30
72.78:0048
75.28:10.68
79.67:0088
82.95:00445
79.23:10229
81.9611.26
ImageNet-100
65.42-+0.36
80.02++0.45
65.80+00.49
76.23++0.37
66.85=+0.19
0.11
in the composition of the unlabeled dataset. For example, when 75%, the unlabeled dataset contains three non-animal classes and one animal classes. refer Appendix [ for further details on the composition of the unlabeled dataset.
The CIFAR- 10/100 task uses CIFAR-10 as the labeled dataset, and CIFAR- as the unlabeled dataset. While the whole CIFAR-100 was taken as the unlabeled dataset, we sampled 100 images per class from CIFAR-10 to simulate the scarce labeled data scenario. Notably, class labels in CIFAR- and CIFAR-100 do not exactly overlap, though there are similarities (e.g., "horse" in CIFAR-10 10 and "cattle" in CIFAR-IOO) Thus, this task complements the SixAnimal task, which had an exact class overlap between labeled and unlabeled data.
The ImageNet-30 task the ImageNet-30 dataset Hendrycks et al. (2019) which a subset of ImageNet limited to 30 classes. Following Saito et al. (2021). we selected 5% of the data from the first 20 classes (approximately 50 samples per class) based on the alphabetical ordering of class names for the labeled dataset, and used the remaining data as the unlabeled dataset.
The ImageNet-1(00 task uses ImageNet- 100 dataset, which sub-sampled 100 classes from ImageNet, as described in Cao et al. (2022). We divided these classes equally into 50% ID and 50% OOD classes following alphabetical order. From each ID class, selected small portion (10%) labeled data with the remaining data forming the unlabeled dataset. This task assesses the effectiveness on higher-resolution images with a greater diversity of classes. Please refer Appendix D for extensive results under various sizes of labeled dataset.
Baseline SSL methods Since we use DWD to transform the unlabeled dataset into a dataset devoid of class distribution mismatch, we comprehensively consider as baseline SSL methods those which operate under the standard setting as well as the open-set setting. The baseline SSL methods under the standard setting are MixMatch (Berthelot et al. 2019). FixMatch (Sohn et al. 2020), and Meta Pseudo Labels (MPL) (Pham al. (021). and the methods under the open-set setting are OpenMatch (Saito)
al. 2021). Fix-A-Step (Huang et al. 2023), and IOMatch (Li et al. 2023).
7

---

5.1 DWD-SL: Labeled Dataset Augmentation
Table ! reports the comparative performance of DWD-SL against various baseline SSL methods. The results clearly demonstrate that DWD-SL substantially surpasses the performance of methods for standard SSL. This suggests that DWD successfully captures both the inherent distribution of the labeled data and the diversity of the unlabeled data, yielding highly effective synthetic labeled data for training.
Notably, DWD-SL even achieves competitive or superior results relative to open-set SSL methods. This advantage stems from DWD-SL's ability to transform the diversity unlabeled data into labeled samples, rather than using this diversity merely form of regularization, seen in baseline SSL methods. For further implementation details, please refer to Appendix
5.2 DWD-UT: Unlabeled Dataset Transformation
Table 2 and T the impact of DWD-UT on the performances of the baseline SSL methods. From the results, we can confirm that DWD-UT effectively addresses the performance degradation caused by the class distribution mismatch. Notably, the most significant improvement was observed in the MPL method. Given that MPL. is based pseudo-laheling, this result indicates that DWD-UT effectively mitigates the confirmation bias associated with pseudo-labeling of OOD instances in the unlabeled dataset.
Figure 3 shows how performance degrades over the range of ( the SixAnimal task. We can clearly that using DWD-UT makes SSL methods generally robust the degree of class distribution mismatch, even though they operate under the assumption that there are no OOD instances in the unlabeled data.
We also remark that DWD-UT further improves the performance of open-set SSI. methods. This implies that DWD- UT is orthogonal to the OOD mitigation mechanisms used in open-set SSL meth- ods, offering distinct contribution in ad- dressing the class distribution mismatch:
Originel unlebeled data
DWD-UT
MixMatch
FixMatch
88
85
Accuracy (96)
84
80
75
83
70
while most of the open-set SSL meth- ods operate under the detect-and paradigm to focus excluding the OOD instances due to their negative impact, the diffusion model provides powerful tool for making up the diversity lost by such exclusion.
65
81
60
50
100
75
100
50
100
Mismatch percentage { (96)
Figure 3: Standard SSL performance with varying
also notable that DWD-UT frequently outperforms DWD-SL, indicating a synergistic effect between DWD-UT and baseline SSL methods. This may be attributed the underlying data selection mechanism the SSL methods (e.g. thresholding used in lecudo-labeling, weighting function in filtering-based methods), which also contribute selectively strengthen the impact of synthetic data.
Additionally, to provide direct evidence of DWD-UT's effectiveness reducing the class distribution mismatch, we compute the minimum distance between each unlabeled data sample and the class centroids of labeled data in the latent space. We refer Appendix E for the results.
5.3 Ablation Studies
We carried out a series of ablation studies on DWD-UT, assessing diffferntt training schemes for the diffuuiin model and varying noise levels for perturbing seed images. For these studies, we employed MPL as the baseline SSL method.
Table shows the results of ablation studies conducted on different training schemes for the dif- fusion model. Several observations can be drawn from these results. Firstly, the utilization of the discriminator to filter or reduce the weight of OOD instances culminates in rather marginal performance improvement. Secondly, the incorporation of the diffusion model to generate synthetic data contributes to a substantial performance surge above the bascline, even when fine-tuned solely
8

---

Table MPL performance using different training schemes. The notation Eo [X] indicates the inclu- sion of component X in finetuning the diffusion model. MPL + d represents that the discriminator utilized for filtering unlabeled data.
Training Method
SixAnimal
Cifar-10/100
ImageNet-30
ImageNet-100
MPL
65.62
70.95
72.65
68.43
MPI. + dæ
67.19
71.73
83.60
70.74
MPL EΦ [Di]
78.70
75.33
87.44
73.38
MPL + E9 (D,, Dul MPL + (D; Du, d6l
80.78
76.79
88.09
74.12
83.88
80.24
90.20
75.66
Table 5: MPL. performance SixAnimal with varying noise levels. DWD-UT's is not applied / 0.
Noise Level
t
200
400
600
= 800
1000
Accuracy (%)
65.62:10.47
73.65
82.03-10.18
8318
83.83:1.111
82.06:10.13
Table 6: Performance of standard SSL and generative augmentation methods on ImageNet-30.
Method
MixMatch
FixMatch
MPL
DPT
DA-Fusion
DWD-SL
Accuracy (%)
68.67:10.7
70.07=0.22
72.65+0.70
78.2840.48
75.26+1.39
82.20:10.28
with the labeled data. This suggests that transforming irrelevant unlabeled data is more effective than simply filtering them out. Lastly, the extra step of fine-tuning the diffusion model with the unlabeled dataset and applying discriminative weighting also offers a nontrivial advantage.
5 the variations in performance with different noise levels during the data transformation process. The results indicate that introducing a moderate level of noise (t 600 800) to the unlabeled data during the forward diffusion process, opposed to initiating from pure Gaussian noise (t = 1000), enhances performance. Therefore, can be inferred from these findings that the efficacy of DWD-UT is contingent upon the balance between the level of noise and the information contained the unperturbed data. We remark that the optimal noise level can differ among data instances. While we fixed the noise level the noise level individually for each sample presents a potential avenue for future research.
600 in our experiments for simplicity, determining
5.4 Comparison with Recent Diffusion-based Augmentation Approaches
We conducted further experiments the ImageNet-30 task compare the performance of DWD with those DPT and DA-Fusion. To ensure fairness in comparison, we equalized the implementation of the model structure, data generation process, and the number of augmented data. As shown in Table 6 both DPT and DA-Fusion have demonstrated effectiveness, yet their performance falls short compared that of DWD. This is because DPT, assuming no distribution shift, sometime generates wrongly labeled synthetic images due to the confirmation bias in pseudo labeld of OOD unlabeled images, and DA-Fusion only augments given labeled samples with subtle visual details (Please refer to appendix[[iee for examples). In contrast, DWD synthesizes new labeled samples by transforming diverse unlabeled data, successfully resolving the distribution mismatch.
5.5 Additional Experiments
We also conducted additional experiments to analyze computational costs, investigate the effect of the number of generated data, and assess syper-paramete sensitivity. For detailed experimental results and analysis, please refer to Appendix

---

Conclusion
In this paper, we highlighted the potential of diffusion models for addressing class distribution mismatch in SSL. We introduced Discriminator-V seeriminattr-Weighheed Diffusion (DWD), a semi-supervised training scheme that leverages discriminator data, facilitating effective training of the diffusion model. Our qualitative and quantitative results demonstrate that DWD captures both the characteristics of labeled data and the diversity of unlabeled data.
identify OOD instances within the unlabeled
Notably, DWD exhibits unique capability to convert irrelevant samples into relevant ones, making it compatible with other SSL methods and illustrating the orthogonality of our approach. Our extensive experiments show that DWD significantly enhances SSL performance in scenarios with class distribution mismatch. We hope that DWD will inspire future research focused on addressing distribution mismatch from a data-centric perspective.
Acknowledgments and Disclosure of Funding
This work was supported by ITTP grant funded by MSIT of Korea (No. RS-2020-11200940, No. RS- 2022-1122031,, No. RS-2019-11190075, No. RS-2024-00397310, No. RS-2024-00343999, No. RS- 2024-00457882), and KAIST-NAVER Hypercreative Al Center.
References
Arazo, E., Ortego, D., Albert, P., O'Connor, N. E., and McGuinness, K. Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning. In 2020 International Joint Conference on Neural Networks (LICNN), pp. 1 -8. IEEE, 2020.
Bachman, P., Alsharif, O., and Precup. D. Learning with Bouudl-nseeblle.. Advances in Neural Information Processing Systems, 27, 2014.
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C. A. MixMatch:
A Holistic Approach to Semi cemi-Supervised Learning. Advances in Neural Information Processing Systems, 32, 2019.
Cao, K., Brbic, M., and Leskovee, J. Open-World Semi-Supervised Learning. International Conference Learning Representations, 2022.
Chapelle, O., Scholkopf, B., and Zien, Eds., Semi-Supervised Learning. IEEE Transactions on Neural Networks, 20(3):542-542, 2009.
Chen, Y., Zhu, X., Li, W. and Gong, S. Semi-Supervised Learning under Class Distribution Mismatch. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3569-3576, 2020.
Christoffel, M., Niu, G., and Sugiyama, M. Class-prior Estimation for Learning from Positive and Unlabeled Data. In Asian Conference on Machine Learning, pp. 221-236. PMLR, 2016.
Cubuk, E. D., Zoph, B., Shlens, J., and Le, Q. V. RandAugment: Practical automated data augmenta- tion with reduced search space. In Proceedings of the IEEE conference computer vision and pattern recognition workshops, pp. 702-703, 2020.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. ImageNet: A large-scale hierar- chical image database. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Du Plessis, M., Niu, G., and Sugiyama, M. Convex Formulation for Learning from Positive and Unlabeled Data. In International Conference on Machine Learning, pp. 1386-1394 PMLR, 2015.
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision, 88(2):303-338, 2010.
Feller, W. On the Theory of Stochastic Processes, with Particular Reference to Applications, 1949.
10

---

Grandvalet, Y. and Bengio, Y. Semi-supervised Learning by Entropy Minimization. Advances in Neural Information Processing Systems, 17, 2004.
Guo, L.-Z., Zhang, Z.. Jiang, Y., Li, Y.-F. and Zhou, Z.-H. Safe Deep Semi- Sani--Spervised Learning for Unseen-Clasn Unlabeled Data. In International Conference on Learning Representations, pp. 3897-3906. PMLR, 2020.
He, K., Zhang. X., Ren, S., and Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
He, R., Han, Z., Lu, X., and Yin, Safe-Student for Safe Deep Semi-Supervised Learning with Unseen-Clan -laess Unlabeled Data. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 14585,14594, 2022.
Hendryeks, D., Mazeika, M., Kadavath, S., and Song, D. Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty. Advances in Neural Information Processing Systems, 32, 2019.
Ho, J., Jain, A., and Abbeel, P. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Huang, J., Fang. C.,, Chen, W., Chai, Z., Wei, X., Wei, P., Lin, L., and Li, G. Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi- Semi-Supervised Learning. In Proceedings of the IEEE International Conference Computer Vision, pp. 8310-8319, 2021.
Huang, 7.,, Yang. J., and Gong, C. They are Not Completely Useless: Towards Recycling Trans- ferable Unlabeled Data for Mlass-Miisatched Suni--upervised Learning. IEEE Transactions on Multimedia, 2022.
Huang, Z., Sidhom, M.-J., Wessler, B., and Hughes, M. C. Fix-A-Step: Semi-supervised Learn- ing from Uncurated Unlabeled Data. In International Conference on Artificial Intelligence and Statistics, pp. 8373-8394, PMLR, 2023.
Jain, S., White, M., and Radivojac, P. Estimating the class prior and posterior from noisy positives and unlabeled data. Advances Neural Information Processing Systems, 29, 2016.
Kiryo, R., Niu, G., Du Plessis, M. C., and Sugiyama, M. Positive-Unlabeled Learning with Non- Negative Risk Estimator. Advances in Neural Information Processing Systems, 30, 2017.
Krizhevsky, A. and Hinton, G. Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto, Toronto, Ontario, 2009.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNer Classification with Deep Convolutional Neural Networks. Communications of the ACM, 60(6):84-90, 2017.
Laine, S. and Aila, T. Temporal Ensembling for Semi-Supervised Learning. In International Conference on Learning Representations, 2017.
LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. Nature, 521((553):436 444, 2015.
Lee, D.-H. Pseudo-Label: : The Simple and Efficient Semi Semi- Supervise Learning Method for Deep Neural Networks. In Workshop challenges in representation learning, ICML, volume 3, pp. 896, 2013.
Li, and Liu, B. Learning to Classify Texts Using Positive and Unlabeled Data. In International Joint Conference on Artificial Intelligence, volume 3, pp. 587-592, 2003.
Li, Z.,, Qi, L., Shi, Y., and Gao, Y. 10Match: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers Utilization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 15870-15879, 2023.
L Lin, T.-Y,, Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft COCO: Common Objects in Context. In Proceedings of the IEEE European Conference on Computer Vision, pp. 740-755, 2014.
11

---

Liu, B., Lee, S., Yu, P. S., and Li, X. Partially Supervised Classification of Text Documents. In International Conference on Machine Learning, volume pp. 387-394, 2002.
Loshchilov, I. and Hutter, F. Decoupled Weight Decay Regularization. In International Conference on Learning Representations, 2019.
Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
McLachlan, G. J. Iterative Reclassification Procedure for Constructing an Asymptotically Optimal Rule of Allocation in Discriminant Analysis. Journal of the American Statistical Association, 70 (350):365-30) 1975.
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. In Advances Neural Information Processing Systems, 2022.
Menon, A., Van Rooyen, B., Ong, C. S., and Williamson, B. Learning from Corrupted Binary Labels via Plass-Probability Estimation. In International Conference Machine Learning, pp. 125-134. PMLR, 2015.
Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8): (188):999-111) 1993, 2018.
Niu, G., Du Plessis, M. C., Sakai, T. Ma, Y.. and Sugiyama, M. Theoretical Comparisons of Dositive-Unlabeled Learning against Positive- Ressaive-Negative Learning. Advances Neural Information Processing Systems, 29, 2016.
Oliver, A., Odena, A., Raffel, C. A., Cubuk, E. D., and Goodfellow, I. Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. Advances in Neural Information Processing Systems, 31, 2018.
Pham, H., Dai, Z., Xie, Q., and Le, Q V. Meta Pseudo Labels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11557-11568, 2021.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. Hiihh-Resolutinn Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Saito, K., Kim, D., and Saenko. K. OpenMatch: Open-set Consistency Regularization for Semi- supervised Learning with Outliers. Advances in Neural Information Processing Systems, 34: 2021.
Sajjadi, M., Javanmardi, M., and Tasdizen, T.. Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning. Advances Neural Information Processing Systems, 29, 2016.
Scudder, H. Probability of eиror of some adaptive pattem-rcccognition machines. IEEE Transactions on Information Theory, 1 11(3):63-371, 1965.
Sohl- Schl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., and Li, C.-L. FixMatch: Simplifying Superviud Learning with Consistency and Confidence. Advances in Neural Information Processing Systems, 33:596-666 608, 2020.
Tarvainen, A. and Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in Neural Information Processing Systems, 30, 2017.
12

---

Trabucco, B., Doherty, K., Gurinas, M. A., and Salakhutdinov, R. Efffective Data Augmentation With Diffusion Models. International Conference on Learning Representations, 2024.
Ward, G., Hastie, T., Barry, S., Elith, J., and Leathwick, R. Presence-only data and the em algorithm.
Biometrics, 65(2):554-563 563, 2009.
You, Z.., Zhong, Y., Bao, F., Sun, J., Li, C., and Zhu, J. Diffusion Models and Semi-Supervisse Learners Benefit Mutually with Few Labels. In Advances in Neural Information Processing Systems, 2023.
Zagoruyko, S. and Komodakis, N. Wide Residual Networks. arXiv preprint wwXiv:1605.071444 2016.
13

---

A Derivation for Proposition 4.1
Problem setting. We consider the two-sample problem setting of PU learning (Ward al. 2009
Niu et al. 2016). The discriminator d(x) aims to solve binary classification problem to classity input data x into negative or positive class y. Let X E IR and Y E (-1,+1) denote the input and output random variables, p(x,y y) be the joint probability density function of (X, Y), and p² (x)
p(x Y = +1), p- (x) p(x x Y = -1) be the positive and negative conditional probability density functions respectively. The labeled and unlabeled data are assume be sampled from p* (x) and p(x) = pp* (x) + μ4 (x) respectively, where = p(Y +1) is class prior.
Proof. The goal of the discriminator can be formulated maximizing following objective
J = Ex Ex~p (x) [log (d (x)) Ex-p (x) log (1 - d (x))]
= p (x) log (d (x)) + p
(x) log (1 d (x)) dx
(10)
First order optimality condition gives
p x
x
(11)
(xx)
d(x)
Rearranging ||
we can easily show that the optimal discriminator (x) satisfies
p (x)
d' d'(x)
(12)
p (x(x) + (x)
Substituting 12 to
8 we have
(x)
p p+(x))
-+((x)
(xx p d (x) + (1 - a) (x))
(13)
p (x) + (1 1pppppp p'(pp(x) (x)
p(x)
Finally, we can use wxx) as importance weights because
pt (xo) p(x))llco(xx,t) - <c|2dx Exo~p co) -c12]
Chooppool((xo)lleo) <1|31
p(xo)
(14)
B Data Generation Process
The data generation process detailed in Algorithm 2 In our experiments, we generate one image per unlabeled image. For the impact of the number of generated data, please refer Appendix F We chose to generate images before the classification task, rather than during each batch iteration of training the classification models. Additionally, while we described the sampling process in DDPM style, DPM-Solve [Lu et al. 2022) utilized in implementation for computational efficiency.
Algorithm 2 DWD Image-Seeded Generation
Input Unlabeled data x" " and diffusion model cg-
Parameter Time step
Sample Gaussian noise: <~ I),
Forward diffusion process to time step t: x² Varx" + v(1 - ā,ee Randomly select class condition c.
# Reverse diffusion process starting at x":
for 1, 1.... do z ~ N(. I) ifi> 1 else =0 x
1-a VI-a,
c (x"",, c, + vT,2
end for Output Transformed data xã and class condition
http://gghhbbboo///uuiiggg////th MIT License
14

---

C Implementation Details
Diffusion model and discriminator. For all experiments, use official implementation of latent diffusion model (Rombach et al. 2022), which is publicly available Since latent diffusion models perform diffusion processes in the embedded latent space, there is a trade-off between computational cost and generation quality depending on the downsampling factor f RRombach et al. (2022). Therefore, we adjust based on dataset scale: f = 2 for small-scale datasets and f for large-scale datasets. The batch sizes for labeled and unlabeled data, denoted as B₁ and B. respectively, are set to B; 16, By 112 for small-scale datasets, and B, 4, B, 12 for large-scale datasets. We follow the Rombach et al. (2022) for other configuration such as learning rate, optimizer, scheduler, etc. After the pre- tre-training phase, we train the models for 200K iterations for Cifar- 10, and IM iterations for ImageNet. For the discriminator, we employ the ResNet- 18 He it al. (2016) followed by 2 MLP layers. We train the discriminator using Adam Loshchilov & Hutter (2019) optimizer with 0.0002 initial learning rate and 0.0001 weight decay. We treat μ as hyper-parameter, and set it within (0.25, 0.33, 0.5). Another hyper-parameter a, which control the balance between labeled and unlabeled, we set 5 in all tasks.
Common configuration for DWD-SL and DWD-UT. To ensure fair evaluation, task-specific settings were established for both DWD-SL and DWD-UT. For tasks associated with Cifar- 10, the Wide ResNet-28-2 architecture Zagoruyko & Komodakis (2016)) was employed, with training conducted using the Adam optimizer at an initial learning rate of 0.03 across 256 epochs 1,024 iterations per epoch. In the ImageNet-30 task, we follow the settings from (Saito et 2l,1) and Li (ct 20.33. Specifically, we employed the ResNet- 18 architecture He et al. (2016), and train for 100 epochs with 1,024 iterations per epoch using AdamW 0.1 initial learning rate.
DWD-SL specific configuration. In all DWD-SL tasks, we maintained a 1:1 ratio between labeled and unlabeled samples within each batch. More specifically, we set B; = B,, = 64 for SixAni and Cifar-10/100, and B; B, 32 for ImageNet-30. We applied RandAugment Cubuk et al. (2020), widely used in the SSL field to achieve robust results, to both original labeled and DWD-SL data. Additionally, applied label smoothing to the cross-entrops loss, following the approach used in MPL Pham et al. (2021). The starting time step t for the reverse diffusion process was set to 600.
DWD-UT specific configuration. For Six and 10/100 tasks, we used B; = 64 and B, = 448, while for ImageNet-30, we used B₁ = 32 and B,, 32. Since ImageNet-30 includes out-of-distribution (OOD) classes in the test set while standard SSL methods inherently cannot identify OOD classes, removed OOD samples from the test set for fair comparison. Additionally, 1OMatch evaluates the performance using balanced accuracy, which classifies all OOD classes as additional single class. This could also be an unfair comparison. Therefore, evaluated only on the closed set for all tasks. Except aforementioned, we follow their papers for -ethod-specific hyper-parameters and setting.
Unlabeled data composition in SixAnimal with varying Ç. We configure the SixAnimal task exactly following [Capellle et al. (2009), Huang et al. (2023). The Table 7 shows the composition of labeled / unlabeled set of SixAnimal task according to mismatch percentage (Ç).
Table 7: Configuration of labeled/unlabeled class mismatch scenario in CIFAR-10 SixAnimal task. The bold text of unlabeled set represent the OOD classes.
Labeled set
Unlabeled set
Ç 25%
Bird, Cat, Deer, Dog. Frog. Horse
Airplane, Dog, Frog. Horse Airplane, Car, Frog. Horse Airplane, Car, Ship. Horse Airplane, Car, Ship, Truck
Ç 50%
Bird, Cat, Deer, Dog. Frog. Horse
75%
Bird, Cat, Deer, Dog. Frog, Horse
100%
Bird, Cat, Deer. Dog, Frog, Horse
bbunss//gthuubcoomm//oo//m//aaas MIT License
15

---

D Extended Evaluation on Class Mismatch and Labeled Data Ratios
Different mismatch ratios of ID and OOD classes in the SixAnimal task. As shown in Figure of our paper, we conducted experiments on the SixAnimal task across various ratios of ID and OOD classes (5 50%, 75%, 100%). To extend these findings, we included additional experiments with lower mismatch ratio, = 25%. Since DWD is designed to address class distribution mismatch, its effectiveness expected to decrease as the mismatch ratio lowers. However, we observed that DWD was still able to improve the baseline method in the 25% case, although the performance gain was relatively diminished.
Table 8: Performance evaluation on the SixAnimal task with various ratios of ID and OOD classes (C).
MixMatch +DWD-UT
FixMatch +DWD-UT
MPL
Mismatch ratio
MixMatch
FixMatch
MPL
+DWD-UT
25%
83,78
85,89 (+2.11)
84.56
85.81 (+1.25)
79.70
84.76 (+ 5.06)
Ç 50%
81.16
84.83 (+3.67)
83.23
87.78 (+4.55)
69.43
85.02 (+15.59)
Ç 75%
80.77
84.72 (+3.95)
82.50
87.17 (+4.67)
65.62
83.88 (+18,26)
(= 100%
79.90
84.08 (+4.18)
81.51
87.03 (+5.52)
57.77
83.73 (+25.96)
Varying the size of labeled data in the ImageNet-100 task. To broaden our evaluation, we conducted additional experiments on the ImageNet- 100 dataset, varying the amount of labeled data. Specifically, we used either 10% or 30% of each ID class as labeled data, with the remaining samples forming the unlabeled set. As shown in Table 9 DWD remains effective with different amounts of labeled data, demonstrating strong performance at both 10% and 30% sampling ratios. Notably, the performance gain is more pronounced at the 10% sampling ratio, as the advantages of data augmentation become clearer with smaller datasets. However, DWD's effectiveness may be constrained when labeled data is extremely limited, as the diffusion model may struggle to accurately represent the labeled data distribution. This limitation is also observed in other generative augmentation methods.
Table 9: Performance evaluation on the ImageNet- 100 task with varying sampling ratio (7))
Sampling ratio
MixMatch
FixMatch
MPL
OpenMatch
Fix-A-Step
1OMatch
DWD-SL
y= 10%
69.30 77.88
65.11 75.83
68.43 71.67
65.42 77.31
65.80 73.90
66.85 76.02
82.81
y= 30%
84.43
MixMatch +DWD-UT
FixMatch +DWD-UT
MPL +DWD-UT
Sampling ratio
MixMatch
FixMatch
MPL
7
69.30 77.88
81.62 (+12.32) 82.26 (+ 4.38)
65.11 75.83
80.38 (+15,27) 81.35 (+ 5.52)
68.43 71.67
75.66 (+7.23)
y = 30%
77.49 (+5.82)
OpenMatch +DWD-UT
Fix-A-Step +DWD-UT
10Match
Sumpling ratio
OpenMatch
Fix-A-Step
10Match
+DWD-UT
> 10%
65.42
80.02 (+14.60)
65.80
76.23 (+10.43)
66.85
80.19 (+13.34)
7 =
77.31
81.50 (+ 4.19)
73.90
78.43 (+ 4.53)
81.52 5.50)
16

---

E Calculating Distance in Latent Space
We extract features (labeled, original unlabeled, transformed unlabeled) from four datasets using pre-trained ResNet-50.. The features were normalized, and the pair- wise Euclidean distances between each unlabeled sample and the nearest class centroid visualized using Gaussian Kernel Density Estimation (KDE). As shown in Figure the minimum distances successfully decrease after DWD's data transformation. This result indicates that using DWD to transform unlabeled data does more than just make it look similar to labeled data; it also makes the data semantically similar in the latent space, showing deeper level of similarity beyond just appearance.
Sixtrimmal
Citar 10100
mageNees:0 30
ImageNec-100 100
10
100
125
110
115
120
1.2
0.00
1.05
135
1:15
1.5
6a5 0.9. 0. 1.0 LAS L3S L15 1.20
5.0 0.00 035 1.0 L2
LB 135
Distaance
Figure 4: The distribution of the minimum distance between each unlabeled data sample and the class centroids labeled data.
F Additional Experiments
Computational cost analysis. The main limitation of our work is the additional computation re- quired by incorporating the diffusion model. On the CIFAR-10/100 task, we measured the wall-clock times and memory consumption for each stage of DWD: pretraining, finetuning (including discrimi- nator training), and sampling. We compare the computation costs with standard SSL methods, and the results are reported in Table TO The additional computation for DWD is not overly burdensome compared to the baselines. It worth noting that while we include pretraining costs for completeness. these can be omitted when off- sfflf pre-trained diffusion models are available.
Table 10: Computational Cost Comparison.
DWD
Baselines
Pretraining
Finetuning
Sampling
Mixmatch
Fixmatch
MPL
Elapsed time (Hours)
13.8
9.7
0.1
9.2
7.4
6.7
Memory (GB)
7.1
8.0
7.0
4.9
5.5
7.4
[Machine specification] GPU: NVIDIA GeForce RTX 3090 Ti, CPU: Intel(R) Core(TM) 19-10980XXE
Effect of the number of generated data. We conducted additional experiments using DWD-SL. on the Cifar- 10/100 task, varying the number of generated data denoted as The results are summarized in the Table 11 Here, N 60K corresponds to the original setting in the paper where one synthetic labeled sample per one unlabeled sample is generated. We observed further improvement at == 120K, with slight deterioration thereafter. Note that similar trend was previously reported in Figure 7(c) of Appendix G in You et al. (2023). A reasonable explanation for the performance deterioration is that excessive value of N could cause the classifier be dominated by synthetic data, thereby neglecting real data, as suggested by You et al. (2023).
Table 11: Effect of the number of generated data
N
6K
30K
60K
120K
240K
Accuracy (%)
77.64
79.17
80.05
81.24
81.07
17

---

Hyper-parameters sensitivity. We conducted additional experiments on the SixAnimal task (<=75%) using DWD-SL assess DWD's sensitivity the hyper-parameters. Again, a serves as weight control the balance between labeled and unlabeled data (Fq. ( and is treated as positive sample ratio of unlabeled data to train discriminator (Eq. 7 in our training scheme. We observed that wide range of a and values successfully outperform most of the baselines (refer to Table I in our paper). Regarding a, an extremely small value may cause the diffusion model training focus excessively and potentially leading to overfitting. Conversely, an extremely large value may cause the training to skew towards the unlabeled data, failing to properly capture the labeled data distribution. In our experiments, an a value around achieves an appropriate trade-off. Regarding μa, the optimal value near the true ratio
the labeled data, failing to reflect the diversity of the unlabeled data
expected.
Table 12: Performance of DWD with various a.
Table 13: Performance DWD with various js.
a
3
5
10
0.125
0.25
0.33
0.55
Accuracy (%)
84.01
85.86
83.83
83.51
Accuracy (%)
84.56
85.86
85.33
84.72
G Images Corresponding to Discriminator's Output
As shown the Figure our discriminator successfully identifies the ID/OOD samples. ImageNet-30 consist of 20 in-domain (ID) classes (acorn, airliner, ambulance, american alligator, banjo, barn, bikini, digital clock, dragonfly, dumbbell, forklift, goblet, grand piano, hotdog, hourglass, manhole cover, mosque, nail, parking meter, pillow) and 10 out-of-domain (OOD) classs (revolver, rotary dial telephone, schooner, snowmobile, soccer ball, stingray, strawberry, tank, toaster, volcano)
(a) High scored samples
(b) Low scored samples
(c) High scored samples
(d) Low scored samples
Figure 5: Selected unlabeled samples based on discriminator's output the SixAnimal (a, b) and ImageNet-30 (c, d).
18

---

H Generated Images
Unlabeled images
Noised images
Transformed images
Labeled images
Airplane
(Rocket)
(Rocket)
Airplane
Airplane
(Lobster)
(Lobster)
Car
Car
Bird
(Rabbit)
(Rabbit)
Bird
(Snail)
(Snail)
Cat
Cat
Dear
(Bear)
(Bear)
Dog
(House)
(House)
Dog
Dog
Frog
(Mushroom)
(Mushroom)
Frog
Frog
(Camel)
(Camel)
Horse
Horse
Ship
(Tank)
(Tank)
Ship
Ship
Truck
(Raccon)
(Raccon)
Truck
Truck
Forward process to time step
Reverse process with class condition "XXX"
XXX
to time step t-1, 0.
Figure 6: Selected Examples of Data Geneartion Process.
19

---

I Generated Images from DPT and DA-Fusion
Unlabeled Image
DPT
DWD
Mosque
(Tank)
(Tank)
Mosque?
Mosque
Forward process time step
XXX
Reverse process with class condition "XXX" time step 1-1, ...
Figure 7: Generated images from DPT and DWD. DPT sometimes generates incorrectly labeled samples (e.g., an image of schooner, which is an OOD class, labeled as mosque). Note that while DPT originally samples images from scratch, we applied our data generation algorithm DPT for comparison.
Original Image
DA-Fusion
Mosque
Mosque
Mosque
Mosque
Mosque
Figure 8: Generated images from DA-Fusion. DA-Fusion only augments given labeled images with subtle visual details.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction reflect the contributions and scope of our work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss our main limitation (additional computation) in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
21

---

Justification:
Please refer Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer: [Yes]
Justification:
We provide open source repository for reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cpippi.ccppulli//uuuia wwwiiicgguie//eeeeeeeeeic CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer Appendix Guidelines:
C
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report experimental results with the standard error.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
23

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : //t::://nnuis//eeeeeeeeeee httppppp......b.....hhc.cccdaeed
Answer: [Yes]
Justification:
We have reviewd the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is direct path any negative application.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cited the original paper that produced the dataset, and provided URL and license of the existing code used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwprspipetattteeeeee thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
25

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide open source repository that is well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We did not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We did involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "8Pt63UTOhP", "reviewer": "Reviewer_KfDe", "summary": "This paper proposes DWD, a new OSSL method that trains a diffusion model to transform OOD unlabeled data to ID images for SSL. DWD can mitigate the class mismatch problem in the OSSL task, which affects SSL performance.", "questions": "For the two usage scenarios of generated data in Sec 4.2, the authors claim that the performance is better when it is served as unlabeled data. This slightly contradicts my intuition. If the generated data is reliable, why does removing labels for training result in better performance?", "limitations": "Please refer to the weakness.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "While previous OSSL methods attempted to distinguish between ID and OOD data through OOD detection training methods, DWD offers a novel perspective by addressing the OSSL problem from the data generation level, which leads to the state-of-the-art OSSL performance.", "weaknesses": "1. The experimental results are expected to be more extensive, for instance, using 1) different ratios of ID and OOD classes and 2) different ratios of labeled and unlabeled data on the same dataset to verify the method’s effectiveness. Such settings are common in recent OSSL methods like [1,2]\n\n2. DWD does not truly enable the model to distinguish between ID and OOD data; instead, it eliminates the OOD samples from the data. Thus, DWD cannot handle potential unseen OOD samples in real-world scenarios.\n\n3. DWD relies on the ratio of OODs in the unlabeled dataset as the prior knowledge to train the discriminator. Such prior knowledge is not always available in real-world scenarios.\n\n[1] Li Z, Qi L, Shi Y, et al. Iomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization, ICCV 2023\n\n[2] Saito K, Kim D, Saenko K. Openmatch: Open-set consistency regularization for semi-supervised learning with outliers, NeurIPS 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thoughtful feedback.\n\n**W1.** We agree that a more extensive set of experiments can help validate DWD’s effectiveness. We thus conducted additional experiments that are common in recent OSSL methods: 1) different ratios of ID and OOD classes, and 2) various sizes of labeled data.\n\n1) Different ratios of ID and OOD classes\n\nAs shown in Figure 3 of our paper, we conducted experiments on the SixAnimal task with different ratios of ID and OOD classes ($\\zeta$ = 50%, 75%, 100%). These experiments demonstrated DWD's ability to enhance baseline SSL performance, under various class distribution mismatch scenarios. To further expand the results, we conducted additional experiments to include  $\\zeta$=25% (below 50%). We report the results in Table A in PDF.\n\nSince DWD is designed to resolve the class distribution mismatch, its effectiveness is expected to decrease with a lower mismatch ratio. However, we observed that DWD was still able to improve the baseline method in the 𝛇 = 25% case, although the performance gain was relatively diminished.\n\n2) Varying the size of labeled data\n\nTo provide a more extensive set of experiments, we conducted additional experiments on the ImageNet-100 dataset, varying the size of labeled data. To build the ImageNet-100 dataset, we sub-sampled 100 classes from ImageNet, as described in [A]. We divided these classes equally into 50% ID and 50% OOD classes following alphabetical order. From each ID class, we selected a small portion (10% or 30%) as labeled data with the remaining data forming the unlabeled set. We report the results in Table B in the PDF.\n\nAs shown in the table, DWD still proves to be effective with varying amounts of labeled data, performing well with both 10% and 30% sampling ratios. Notably, the performance gain is greater with the 10% sampling ratio compared to the 30% ratio, as the benefit of data augmentation is more pronounced when the dataset is smaller. However, it is important to note that DWD’s effectiveness is expected to be marginal when the amount of labeled data is extremely small, as the diffusion model struggles to accurately capture the labeled data distribution. We remark that this limitation is also common to other generative augmentation approaches. \n\n\n**W2.** Although we didn’t explicitly handle OOD samples in the paper, it is important to note that our approach involves training a separate discriminator to distinguish between ID and OOD data during the diffusion model training. By integrating this discriminator with the trained classifier, we can straightforwardly reject OOD samples and only classify ID samples during inference time.\n\n\n**W3.** We remark that we did not assume any prior knowledge of $\\mu$ in our experiments. Instead, we treated $\\mu$ as a hyper-parameter and systematically tuned it using a validation set, following the standard protocol for hyper-parameter tuning in the SSL literature.\n\n\n**Q1.** We understand the possible ambiguity introduced by the last sentence in Section 4.2. The sentence can be interpreted either way: (1) DWD-UT demonstrates better results than DWD-SL and/or (2) applying DWD-UT to baseline SSL methods improves the baseline performance. We intended to claim the latter rather than the former interpretation. We will carefully rephrase the corresponding sentence to mitigate this ambiguity.\n\nStill, as discussed in Section 5.2, we found that utilizing the DWD-UT with baseline SSL methods outperformed DWD-SL in some cases. We suspect this is because there are variations in the qualities of the generated images, since they are generated without considering the relevance between the class condition and the guide image. If the class condition is highly relevant to the guide image (e.g. very similar class), a high-quality image would be generated; otherwise, the quality of the generated image can be relatively low. When DWD-UT is combined with sophisticated data selection mechanisms commonly present in SSL methods (e.g., thresholding used in pseudo-labeling, weighting functions in filtering-based methods), it can discern high versus low quality samples, thereby enhancing performance compared to DWD-SL. \n\n**Reference**\n\n[A] K Cao et al., “OPEN-WORLD SEMI-SUPERVISED LEARNING.”, ICLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n6OriUkB0m", "reviewer": "Reviewer_9H8V", "summary": "The paper proposes an approach that leverages a diffusion model to enrich labeled data using both labeled and unlabeled samples to try to solve the traditional ssl method struggling in the real-world scenarios, i.e., a large number of irrelevant instances in the unlabeled data that do not belong to any class in the labeled data. Specifically, authors combine diffusion model training with a discriminator, and convert an irrelevant instance into a relevant one. Empirically, the data augmentation approach which is proposed by this paper significantly enhances the performance of SSL methods.", "questions": "1. As mentioned in Weakness 2, can the method be applied effectively to larger datasets?\n2. Can this method be applied to other generative model, such as GAN, Flow?\n3. How about the generation performance? can author evaluate the common generation metric such as FID, IS?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The motivation is clear, and the method is effective and easy to follow.\n2. The combination of DWD-UT and semi-supervised methods is interesting, and the results are impressive.\n3. The results in the appendix address many of my questions, and the analysis is comprehensive.", "weaknesses": "1. There is a lack of analysis regarding the structure and training methods of the diffusion model networks, such as DiT and DDPM. Is DiT also effective for DWD-UT and DWD-SL?\n2. The ImageNet-30 dataset is small. It remains to be verified whether the method can perform well in an open semi-supervised setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive feedback.\n\n**W1.** Thank you for bringing up the different diffusion model network structures and training methods for further improving our work. Given that DPT [A], which is closest prior work to our methodology, has demonstrated a remarkable performance with U-ViT [B], we anticipate that DiT [C], which is also based on ViT, will perform well with DWD. However, due to current limitations in computational resources and time, we plan to explore this aspect in future research. \n\n**W, Q1.** To verify DWD’s scalability to larger datasets, we conducted additional experiments on the ImageNet-100 dataset, which is larger than ImageNet-30. To construct the ImageNet-100 dataset, we sub-sampled 100 classes from ImageNet, as described in [D]. We divided these classes equally into 50% ID and 50% OOD classes, following alphabetical order. From each ID class, we selected a small portion (10% or 30%) as labeled data, with the remaining data forming the unlabeled set. We report the results in Table B in the PDF.\n\nAs shown in the table, DWD-SL outperforms the baselines, and DWD-UT significantly enhances baseline performance. From these results, we can conclude that DWD can be applied effectively to larger datasets. \n\n**Q2.** Our training scheme, which uses the discriminator to obtain importance weights for unlabeled data, can be applied to other generative models since such importance sampling does not impose any specific restrictions on the types of generative models. However, since the proposed data generation method starts from a partially noised unlabeled image, it cannot be applied to generative models whose generation process does not involve a noisy image as an intermediate step.\n\n**Q3.** We acknowledge the importance of validating generation performance. We therefore measured the FID and IS scores of the generated samples. Additionally, we included intra-cluster pairwise LPIPS distance [E] which represents the degree of overfitting, as FID and IS scores might not capture overfitting issues in domains with limited data. To investigate whether DWD improves generation performance, we assessed these three scores under different training schemes: (a) with labeled data only, (b) with both labeled and unlabeled data using the objective (6) in our paper, and (c) with DWD.\n\n|                           | FID score (↓) | IS score (↑) | LPIPS (↑) |\n|---------------------------|---------------|--------------|-----------|\n| (a) Labeled data only     | 60.1          | 21.4         | 0.167     |\n| (b) Labeled and unlabeled | 69.2          | 20.4         | 0.170     |\n| **(c) DWD**               | **43.6**      | **23.9**     | **0.177** |\n\nWe observed that case (b) results in a better intra-cluster pairwise LPIPS distance but worse FID and IS scores compared to case (a). This implies that while utilizing unlabeled data helps mitigate overfitting problems, irrelevant unlabeled data makes it difficult for the diffusion model to learn the distribution of the labeled data. On the other hand, DWD achieves the best scores across all three metrics. This indicates that DWD effectively addresses both overfitting issues and the problems caused by irrelevant unlabeled data, leading to better generation performance.\n\nWe appreciate the suggestions for more diverse validation of our method, and hope that the response above addresses your comments.\n\n\n**References**\n\n\n[A] You et al., “Diffusion models and semi-supervised learners benefit mutually with few labels”, NeurIPS, 2023.  \n[B] Bao et al., “All are worth words: a vit backbone for score-based diffusion models”, NeurIPS Workshop on Score-Based Methods, 2022.  \n[C] Peebles et al., “Scalable Diffusion Models with Transformers”, ICCV, 2023.  \n[D] K Cao et al., “OPEN-WORLD SEMI-SUPERVISED LEARNING.”, ICLR, 2022.  \n[E] Ojha et al., “Few-shot Image Generation via Cross-domain Correspondence”, CVPR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9och363Cv2", "reviewer": "Reviewer_5EHv", "summary": "This paper aims to address the challenge of utilizing unlabeled data in SSL, especially when there is a mismatch in class distributions between labeled and unlabeled data. The authors propose to leverage diffusion models to convert unlabeled data, especially out-of-distribution data, into in-distribution samples. Besides, a discriminator is jointly trained to filter out potentially irrelevant data. Experiments on image classification tasks show that the proposed method can greatly improve the performance of existing SSL methods, especially in scenarios with large class distribution mismatch.", "questions": "1. The computation cost for different generative augmentation methods can be varied. How do you ensure the comparisons are fair?\n2. How does the number of labeled data affect the effectiveness of the proposed method?", "limitations": "Some limitations are discussed in Appendix.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper proposes a novel approach to tackle the class distribution mismatch problem in SSL by leveraging diffusion models to convert unlabeled or OOD samples to in-distribution samples. This is a creative combination of ideas from generative modeling and SSL and is well-motivated.\n\n2. The results demonstrate clear improvements over state-of-the-art SSL methods, especially in challenging scenarios with large distribution mismatch. The proposed method can also be served as a plug-in approach to enhance existing SSL methods.\n\n3. The paper provides good visualizations that help understand the generated samples and the effectiveness of the method and discriminator's scores.", "weaknesses": "1. The use of diffusion models introduces significant computational costs compared to standard SSL methods. While this is acknowledged, a more detailed analysis of the trade-offs between performance gains and computational cost would be valuable.\n\n2. More comparisons with related generative augmentation in SSL are needed. The paper only compares with the two closely-related work, DPT and DA-Fusion, on one dataset (ImageNet-30). Comparisons to advanced data augmentation techniques used in SSL on more datasets would provide more context.\n\n3. The experiments are limited to relatively two~three small-scale datasets (The SixAnimal dataset is sampled from CIFAR10). It's unclear how well the approach would scale to larger, more complex datasets.\n\n4. The paper focuses primarily on successful cases. A discussion of scenarios where DWD might not work well or potential limitations would provide a more balanced view.\n\n5. While the discriminator is a key component, the presentation of the discriminator and the related positive-unlabeled learning is a bit confusing. More explanation on how the discriminator is trained and why the PU learning is adopted would be helpful. Also, have you considered alternative designs for the discriminator? \n\n6. The method introduces several hyper-parameters, such as the $\\alpha$, $\\mu$. A more thorough discussion or analysis of the sensitivity to these parameters would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive feedback.\n\n**W1.** To investigate trade-offs between performance gains and computational cost, we conducted additional experiments on the CIFAR-10/100 tasks, varying U-Net depths $d$ (i.e., the number of residual blocks per downsample) in the diffusion model. The results are summarized in the table below:\n\n|                                    | $d = 1$  | $d = 2$ | $d = 4$   |\n|------------------------------------|----------|-------------------------------|-----------|\n| **FixMatch + DWD-UT Accuracy (%)**    | 80.7     | 83.8                          | 84.2      |\n| **Model parameters (M)**           | 50.0     | 70.1                          | 110.2     |\n| **Elapsed time (Hours)**           | 15.6     | 21.5                          | 32.3      |\n\nAs expected, increasing the depth of the U-Net led to further performance improvements. Thus, by adjusting the depth according to available computational resources, one can effectively balance the trade-off between model performance and computational cost.\nAdditionally, the architecture of diffusion models (e.g., U-ViT [A], DiT [B]) can also influence computational cost and performance, which we plan to explore in future work.\n\n**W2.** First of all, we would like to highlight that DPT and DA-fusion demonstrate state-of-the-art performance in generative data augmentation techniques. To the best of our knowledge, other existing generative approaches based on VAE [C] or GAN [D] are limited to low-resolution data, which is why we did not include them as baselines. Additionally, we remark that some of our baselines, such as Fixmatch, Mixmatch, and Fix-A-Step, can be also considered as data augmentation techniques. Therefore, our experimental results demonstrate that DWD outperforms the well-known data augmentation techniques used in SSL. If you could suggest a manageable set of other advanced augmentation methods in SSL, we would be more than happy to conduct further experiments during the author-reviewer discussion period.\n\n**W3, Q2.** To verify DWD’s scalability to larger datasets and investigate the impact of the size of labeled data on DWD, we conducted additional experiments on the ImageNet-100 dataset, varying the size of labeled data. To construct the ImageNet-100 dataset, we sub-sampled 100 classes from ImageNet, as described in [E]. We divided these classes equally into 50% ID and 50% OOD classes, following alphabetical order. From each ID class, we selected a small portion (10% or 30%) as labeled data, with the remaining data forming the unlabeled set. We report the results in Table B in the PDF.\n\nAs shown in the table, DWD-SL outperforms the baselines, and DWD-UT significantly enhances baseline performance. From these results, we can conclude that DWD can be effectively applied to larger datasets. \nIn addition, DWD proves effective with varying amounts of labeled data, performing well with both 10% and 30% sampling ratios. Notably, the performance gain is greater with the 10% sampling ratio compared to the 30% ratio, as the benefit of data augmentation is more pronounced when the dataset is smaller. However, it is important to note that DWD’s effectiveness is expected to be marginal when the amount of labeled data is extremely small, as the diffusion model struggles to accurately capture the labeled data distribution. We remark that this limitation is also common to other generative augmentation approaches. \n\n**W4.** We acknowledge the importance of discussing potential limitations to provide a more balanced view. To investigate scenarios where DWD might not perform well, we extended Figure 3 in our paper to include a lower degree of class distribution mismatch ($\\zeta$=25%). We report the results in Table A in the PDF.\n\nSince DWD is designed to resolve class distribution mismatch, its effectiveness is expected to decrease with a lower mismatch ratio. However, we observed that DWD was still able to improve the baseline method in the $\\zeta$ = 25% case, although the performance gain was relatively small.\n\nAs previously discussed in response to [W3,Q2], another limitation of DWD is its limited applicability with extremely small size of labeled data. Although DWD can benefit from the diversity of unlabeled data, it must first learn the distribution of the labeled data.\n\n**W6.** We conducted additional experiments on the SixAnimal task ($\\zeta$ = 75%) using DWD-SL to assess DWD's sensitivity to the hyper-parameters.\n\n1)  **$\\alpha$** : We varied  across values of {1, 3, 5, 10}, and the results are presented in the table below:\n\n|                | $\\alpha = 1$   | $\\alpha = 3$ | $\\alpha = 5$  | $\\alpha = 10$ |\n|----------------|----------------|--------------|---------------|---------------|\n| Accuracy (%)   | 84.0           | 85.9         | 83.8          | 83.5          |\n\n2) **$\\mu$** : We varied  across values of {0.125, 0.25, 0.33, 0.5}, and the results are presented in the table below: \n\n|                | $\\mu = 0.125$   | $\\mu = 0.25$ | $\\mu = 0.33$ | $\\mu = 0.5$ |\n|----------------|-----------------|--------------|--------------|-------------|\n| Accuracy (%)   | 84.5            | 85.9         | 85.3         | 84.7        |\n\nWe observed that a wide range of $\\alpha$ and $\\mu$ values successfully outperform most of the baselines (refer to table 1 in our paper).\n\nRegarding $\\alpha$, an extremely small value may cause the diffusion model training to focus excessively on the labeled data, failing to reflect the diversity of the unlabeled data and potentially leading to overfitting. Conversely, an extremely large value may cause the training to skew towards the unlabeled data, failing to properly capture the labeled data distribution. In our experiments, an $\\alpha$ value around 3 achieves an appropriate trade-off.\n\nRegarding $\\mu$, the optimal value is near the true ratio $1-\\zeta$, as expected.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## **General response**

We appreciate all the reviewers for taking the time to provide constructive feedbacks on our paper. We are very encouraged that the reviewers have recognized the following strengths in our work:
1) Proposition of a novel perspective by addressing the class distribution mismatch problem at the data generation level. (Reviewer KfDe, 5EHv)
2) Presentation of a creative methodology with clear motivation. (Reviewer 5EHv, 9H8V)
3) Demonstration of strong empirical performance. (Reviewer KfDe, 9H8V, 5EHv)
4) Provision of effective visualizations that aid in understanding. (Reviewer 5EHv)
5) Inclusion of comprehensive analysis with additional experiments in the appendix. (Reviewer 9H8V)

Below, we summarize the main concerns raised:
1) Necessity of assessing DWD on larger datasets. (Reviewer 9H8V, 5EHv)
2) Necessity of evaluating DWD with different ratios of ID and OOD classes and varying sizes of labeled data. (Reviewer KfDE, 5EHv)
3) Lack of analysis on hyper-parameter sensitivity, generation performance, and trade-offs between performance gains and computational cost. (Reviewer 9H8V, 5EHv)

We have addressed the comments with individual responses. If you have any questions or require further clarification, please let us know, and we will be glad to address them during the discussion period.

Additionally, please refer to our one-page PDF, which includes the following results:
* Table A : SixAnimal task with different ratios of ID and OOD classes.  (Reviewer KfDE, 5EHv) 
* Table B : ImageNet-100 task with varying sizes of labeled data. (Reviewer KfDE, 9H8V, 5EHv)

Meta-review: This paper focuses on the problem of open-set semi-supervised learning, where there is a large number of irrelevant instances in the unlabeled data that do not belong to any class in the labeled data. To solve the open-set semi-supervised learning problem, this paper proposes a data-centric generative augmentation approach that leverages a diffusion model to enrich labeled data using both labeled and unlabeled samples. Comprehensive experimental results demonstrate the effectiveness of the proposed approach.

After the authors' rebuttal, all the reviewers gave positive ratings to this paper (i.e., 3 x Weak Acceptance) and recognized that the proposed approach is novel and the performance is satisfactory. Therefore, I recommend accepting this paper as a poster presentation.

Please include the additional experimental results of the SixAnimal task and the ImageNet-100 task in the final version to further improve this paper.
Decision: Accept (poster)