Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation
Authors: Hiroshi Takahashi
Tomoharu Iwata
Atsutoshi Kumagai
Yuuki Yamanaka
Tomoya Yamashita
Venue: ICLR 2025 Poster (2025)
Keywords: Diffusion Model
Positive-Unlabeled Learning
Primary area: generative models
TL;DR: We prevent diffusion models from generating sensitive data by using positive-unlabeled learning.

Abstract: Diffusion models are powerful generative models but often generate sensitive data that are unwanted by users,
mainly because the unlabeled training data frequently contain such sensitive data.
Since labeling all sensitive data in the large-scale unlabeled training data is impractical,
we address this problem by using a small amount of labeled sensitive data.
In this paper,
we propose positive-unlabeled diffusion models,
which prevent the generation of sensitive data using unlabeled and sensitive data.
Our approach can approximate the evidence lower bound (ELBO) for normal (negative) data using only unlabeled and sensitive (positive) data.
Therefore, even without labeled normal data,
we can maximize the ELBO for normal data and minimize it for labeled sensitive data,
ensuring the generation of only normal data.
Through experiments across various datasets and settings,
we demonstrated that our approach can prevent the generation of sensitive images without compromising image quality.

Published as a conference paper at ICLR 2025
POSITIVE-UNLABELED DIFFUSION MODELS FOR PREVENTING SNSSTIIVE DATA GENERATION
Hiroshi Takahashi
Tomoharu Iwata
Atsutoshi Kumagai NTT Corporation
NTT Corporation
NTT Corporation
Yuuki Yamanaka NTT Corporation
Tomoya Yamashita
NTT Corporation
ABSTRACT
Diffusion models are powerful generative models but often generate sensitive data that are unwanted by users, mainly because the unlabeled training data frequently contain such sensitive data. Since labeling all sensitive data in the large-scale unlabeled training data is impractical, we address this problem by using a small amount of labeled sensitive data. In this paper, propose positive-unlabeled diffusion models, which prevent the generation of sensitive data using unlabeled and sensitive data. Our approach can approximate the evidence lower bound (ELBO) for normal (negative) data using only unlabeled and sensitive (positive) data. Therefore, even without labeled normal data, for normal data and minimize it for labeled sensitive data, ensuring the generation of only normal data. Through experiments across various datasets and settings, we demonstrated that our approach can prevent the generation of sensitive images without compromising image quality.
maximize the ELBO
INTRODUCTION
Diffusion models (Ho et al. 2020 Sohl-Dickstein et al. 2015 Song & Ermon 2019 Song et al.) 2020b) are powerful generative models that have become the de facto standard, and are applied to various fields such images (Dhariwal & Nichol 1 Rombach et al. 2022). audio (Chen al.
2020 Kong et al. 2020) et al. 2021). and text (Austin et al. 2021 Li et al. 2022| Gong ct al,
2022 The training of diffusion models can be regarded as the maximization of the evidence lower
bound (ELBO), which is the tractable lower bound of the log-likelihood, on the training data (Ho)
cet al. 2020). Users collect these training data from sources like the internet to generate the contents they want, and then perform either training from scratch or fine-tuning.
Unfortunately, diffusion models have the potential generate inappropriate, discriminatory, or harmful contents that are unwanted by users (Brack et al., (022). For example, they might generate sexual images of real individuals (Mirsky & Lee [021]] Verdoliva 20200). We refer to such contents as sensitive data. The primary cause of this problem is that such sensitive data are included in the training data. To handle this problem, existing approaches have applied data forgetting (Gandikota et al. 2023) Zhang et al., 2024) and continual learning (Heng & Soh, 2024) to pre-trained diffusion models. For example, to forget sensitive data, Heng Soh (2024) first prepare normal and sensitive data, then attempt to maximize the ELBO for the normal data while minimizing it for the sensitive data. This approach requires a lot of normal data that do not include any sensitive data. However, it is difficult to prepare such normal data. When using training data collected from the internet, it is difficult to manually remove all sensitive data. In addition, when using pre-trained models to generate training data, unintended sensitive data may be produced. That is, what we can actually use is not clean normal data, but unlabeled data that contain both normal and sensitive data. On the other hand, it is easy to prepare small amount of sensitive data that users may not want. Hence, we need to train diffusion models in positive-unlabele (PU) setting (Du Plessis et al. 2014 2015) Kiryo et al., 2017). where we have access only to unlabeled and sensitive (positive) data, but not to normal (negative) data.
Corresponding author; hi rosh bm. takahashi com

---

Published as a conference paper at ICLR 2025
6
4
0
2 4
0
4
2
5
5
3
1
3
8 8
4
4
4 6 8
6 8
4
V
4
8 2 4 2
6
6
F
2
4
4
9
3
1
7
5
7
8
4 4 2
0
0
4
0 2
4 4 0 O
8
4 4
2
4
4
6
8
7
6 a
6 2
0
8
6
0
9
/
7
3
5
6
4
8 8 2 0 4
6
6
8
8 4
0
4
0
63
1
1371
3
3
0
8
5 4
2 8
0
7
2
5
0
7
173%
9
3
8
4
8
5
8 2
4
8 9 8
0
4
000
1 7
$
3
7
9
2
8
8
2
2
1 7
5
7
7
0
4
9
8
4
0
0
O 2
2 6
2 2
9 5
9
3
0 O
6
0
4 2
D
(a) Unlabeled training data. (b) Sensitive training data. (c) Unsupervised samples.
(d) Proposed samples.
Figure 1: MNIST examples by the proposed method, where even numbers are considered normal data and odd numbers considered sensitive data. (a) The unlabeled training data contain 10% sensitive data (odd numbers). (b) The sensitive training data contain only odd numbers. (c) When the diffusion model is trained in the standard way using the unlabeled training data, the generated samples include sensitive data (odd numbers). (d) When the proposed method is applied to the diffusion model, generates only normal data (even numbers).
In this paper, propose positive-unlabeled diffusion models, which prevent diffusion models from generating sensitive data for this PU setting. We model an unlabeled data distribution with a mixture of normal and sensitive data distributions. Accordingly, the normal data distribution as mixture of unlabeled and sensitive data distributions. With this trick, we approximate the ELBO for normal data only using unlabeled and sensitive data. Therefore, even without labeled normal data, we can maximize the ELBO for normal data and minimize it for labeled sensitive data. Note that our approach requires only small amount of labeled sensitive data because such data are less diverse than normal data.
be rewritten
Figure D shows MNIST examples by unsupervised and proposed methods. In these examples, even numbers are considered normal data, while odd numbers are considered sensitive data. Our approach can maximize the ELBO for normal data (even numbers) and minimize it for sensitive data (odd numbers), using only unlabeled and sensitive data. As result, the diffusion model trained with our approach generates only normal data (even numbers).
Our approach can be applied to training from scratch as well to fine-tuning by using the parameters of a pre-trained model as initial values. Through experiments across various datasets and settings, demonstrated that approach effectively prevents the generation of sensitive images without compromising image quality.
Our approach also be easily extended ceattive-negative--nllaa a small amount of normal data. Our approach is well-suited for crivac-preesevvnn applications, ensuring that sensitive samples The code is available at O hhattps///gtthbbb.cc//aaaaaaaaaa
setting, where can
inadvertently generated while maintaining sample quality. /pudm
Our contributions be summarized as follows:
We propose positive-unlabeled diffusion models, designed to prevent diffusion models from generating sensitive data when only unlabeled and sensitive data are available.
Our approach supports both training from seratch or fine-tuning pre-trained models.
Experiments on various datasets show that our approach effectively prevents the generation of sensitive images without compromising image quality.
2 PRELIMINARIES
2.
PROBLEM SETUP
First, we describe our problem setup. Given unlabeled dataset 24 = fx(1).... x(N) and labeled sensitive dataset S = (x(").... M( for training, 24 contains both normal and sensitive data points. Our goal is train diffusion models so that they generate only normal data using 'd and S.
2

---

Published as a conference paper at ICLR 2025
2.2
DIFFUSION MODELS
Next, we review diffusion models (Ho et al. 2020 Sohl-Dickstein et al., 2015 Song & Ermon Song et al. 2020b). They consist of two processes: diffusion and denoising.
2019
The diffusion process gradually adds Gaussian noise to the original data point X over T steps, eventually converting it into standard Gaussian noise xT. The denoising process gradually removes the noise from X7. converting it back to the original data point Xo. The noisy data points XI,...,XX XT can be viewed as latent variables of the same dimension the original data point Xp.
They model the probability of data point X using latent variables X,,..., XT as follows:
po (xu) = / po(xu:1 dxxI:T
(1)
where pe represents the denoising process:
T Po (xo.r) = p(xT) II (x (x.)-
(2)
These distributions are modeled by Gaussian distributions:
Pa xx x X+) = (x 1 (), Eo(x,, ()), p(XT) = (XT;; 0,1).
(3)
Here, (X, t) and (x, t) are neural networks with parameter 6, which estimate the mean and covariance matrix of the Gaussian distribution, respectively. (xT;; I,I) is the standard Gaussian, where 0 denotes the zero vector and denotes the identity matrix.
On the other hand, the diffusion process is modeled by Gaussian distributions follows:
T
q(xITTX))
II ((x/(x=-))
q(x:|x=-1) = Nxxx; (1-
(4)
where E (0, 1) is monotonically decreasing hyperparameter. A notable property of the diffusion process is that x, an arbitrary step can be sampled in closed form:
q(x (xo) = Nxxx /,200 (1 oz I)).
0g
ae-
(5)
By applying the reparameterization trick Kingma 2013). X₁ can be rewritten as follows:
X2
v م, + /1 āgc,
(6)
where e is a sample drawn from a standard Gaussian A (e;;0,1
With these distributions, the ELBO for each data point xo. (xo 0), can be derived follows:
po(xo:)) (xxxx1(Xxx
log po(xo:)) g(x1:T (xo)
log po (Xn) = log E. (x(
2 (g(xerl))))
(Xu: 0),
(7)
where E[|] is the expectation. This ELBO can be simplified to the following objective function:
- LDpM (Xp; 0) x Eu(().()) [||c- co (Vāixo + vI- ti,,,)))) +)]
e(xo: O),
(8)
where u(t) is uniform distribution over 1 to T, p(e) is standard Gaussian Nee 0, and is neural network that estimates the noise e from X4 Vopya + VI- dic at step f. That is, we can train diffusion models by minimizing the squared error between the noise and the estimated noise e (xt,,)) t). Note that the ELBO is maximized by minimizing {(x;; 0), and conversely, the ELBO is minimized by maximizing it.
In standard diffusion model training, all unlabeled data points in 24 are assumed to be normal, and the following objective function minimized:
Σ ((x (x))))))
(9)
However, since the unlabeled data often contain sensitive data, the model trained with the above objective may generate sensitive data.
3

---

Published as a conference paper at ICLR 2025
3 PROPOSED METHOD
In this section, we propose positive-unlabeled diffusion models, preventing diffusion models from learning sensitive data using U/ and 8. Our approach is based on PU learning that aims to train a binary classifier to distinguish between positive and negative data using positive and unlabeled data (Du Plessis al., 2014 2015 Kiryo et al.) 2017). Hereafter, we will also refer to normal data points as negative (-) samples and sensitive data points as positive (+) samples.
3.1 SUPERVISED DIFFUSION MODELS
First, assuming that both normal and sensitive data are given, we discuss a supervised approach for training a diffusion model. For example, as described in (Heng & Soh 2024), we can simply minimize 8(x; 0) for the normal data and maximize for the sensitive data. However, since {(x; 0) is bounded below but unbounded above, maximizing f(x; 0) will cause it diverge to infinity, resulting in meaningless parameters. Although 1/E(x; 9) can be minimized as an alternative, this requires additional hyperparameter that balances f(x; 0) and 1/((x: 0), described in (Ruff)
et al. 2019).
As reasonable supervised approach, following Yamannaa et al. 2019), introduce binary classification framework into diffusion model training. Let y = represent normal data and represent sensitive data. We model the conditional probability po using ((x; 0) follows:
[exp(=((x;)))
yy -0 0)
po(ylx)= [1
(10)
1)=
small E(x; 0) leads higher po (y = 0fx), while large {(x; f) results in a higher po (y 1|x).
With this conditional probability, we introduce the binary cross entropy as the loss function for each data point as follows:
( y; 0) = - logpo(ulx)) = (1 v) 0) ylgg(1 expp exp(- 0)))
(11)
This loss function minimizes f(x; 0) for y = O. and maximizes it for y = 1 thorough minimizing - log(1 exppeexp--xx> 0))). Since these two terms are bounded below, there is no risk of divergence.
In addition, as described in (Yamanaka et al. 2019), they are well-balanced, climinating the need for additional hyperparameters. Further theoretical justification is provided in Appendix.
In a supervised approach, we assume that all unlabeled data points i are normal, and minimize the following objective function:
1 N
( 0:00)
1 + M m==1
Σ
BBC (x 1)
(12)
This supervised approach is expected to perform better than the unsupervised approach (Eq. ( because it can handle sensitive data S. However, the presence of sensitive data in the unlabeled data weakens the effect maximizing f(x; 0) for sensitive data.
3.2 POSITIVE-UNLABELLI DIFFUSION MODELS
To handle the unlabeled data 'I that contain sensitive data, introduce PU learning framework into the supervised diffusion model. Let (xx(x) represent the unlabeled data distribution, (s(x) represent the sensitive data distribution, and (x(x) represent the normal data distribution. The datasets U and are assumed to be drawn from pir (x) and (x),,, respectively. We model (x((x) with mixture of (x) and (x):
(x) = 3xs + (1 - (x),
(13)
where E [0, represents the ratio of the sensitive data in the unlabeled data U. Accordingly, px((x) can be rewritten as follows:
(1 (1- (xx))x(x) per (x)
(14)
Note that we can estimate the hyperparameter 33 by PU learning approaches (Menon et al. 2015
Ramaswamy et al. 2016 Jain et al. 2016 Christoffel al. 2016 NNkajjmaa & Sugiyama 2023).
4

---

Published as a conference paper at ICLR 2025
Algorithm Positive-Unlabelee Diffusion Model with Stochastic Gradient Descent
Require: unlabeled and sensitive datasets (4,, hyperparameter 3 [0, 1]
Ensure: parameter of diffusion model 0
1: while not converged do
2: Sample mini-batch B from datasets (4,S)
3: Compute L (f), Zu (f), and Lg (0) Eq.
18 with
4:
if L (0) 3Cg (0) 2 then
5:
Compute gradient Vo(3 + (T (0) - (L)
6: else
7:
Compute gradient V BC3
8:
end if
9: Update with the above gradient
end while
we have access to PN (x), we can train the diffusion model by minimizing the following supervised
objective function:
Cpx (0) Prs (fnCE(x, 1; + (1 By [(CCE(X, 0; 0)
(15)
Since we do not have access to (x(()) in practice, we compute the second term in Eq. Eq. (14) as follows:
15 using
(1 - 1-3) (xB 0; 0)] = Ep [(BCE (x, 0; 0)] pp (CBE 0,0)).
(16)
Hence, only using (x) and ps (x)... we can compute LP (0) as follows:
CPN (0) = ps (x 1; 0)] + Epu (x 0; 0)] Es BBBE 0)0]]
(17)
We approximate (PN(0) using the datasets U and follows:
M Σ
(CC (m) ,0;0) (nc))x( 1 M
(PN(0) =
(m)) (m)) 11;0 +
Σ M
BCC ( (m)),,00))
(0)
(0)
4s(0) (0)
(18)
In Eq. 18 30) is the approximation of (1-3)lEpx (BBE (x, 0, 0,0)] 0. Unfortunately.
Cu (0) 3Cg (0) 2 0 does not always hold, which can lead to over-fitting (Kryyo et al.
2017). To
avoid this, our training objective function ensures Lu (0) 3 (0) 2 0 according to (017) as follows:
Kiryo et al.
(P) (0) + max {0, -0C
(19)
We optimize this objective using the stochastic optimization method such as Adam (Loshchilov) 2017 In practice, when E5 according to Kiryo et al. (2017). Algorithm Provides the pseudocode for our approach.
(
we maximize it until (ū( (C5
2 0,
Alternatively, instead of the max value max {0. -2)
Eq. 19 we can use the
absolute value (0) 355 (f) according to (Hammoudeh & Lowd 2020
This is effective for more complex models such as Stable Diffusion (Rombach 2022 shown in Appendix.
3.3 EXTENSION TO CONDITIONAL DIFFUSION MODELS
While the above discussion has focused on unconditional diffusion models, our approach can be casily extended conditional diffusion models. Given condition с such text, is sufficient to replace the ELBO in Eq. (8) with the following ELBO:
f {xx,, c; 0) Bu(t))a(e) [||= eo (Vaxo + VI Vi-āaa c,e,
(20)
where is neural network that estimates the noise from x; Varxa VI- āic, condition e and step t. We can efficiently draw samples from this conditional model by using classifier guidance (Ho & Salimans 2022).
5

---

Published as a conference paper at ICLR 2025
4 RELATED WORK
4.1 ERASING SENSITIVE CONCEPTS DIFFUSION MODELS
Several approaches have tried to prevent conditional diffusion models from generating sensitive
images (Brack et al., 2022 Gandikota et al., 2023 Zhang et al., 2024 Heng & Soh 2024). For example, Henn & Soh| (2024) aim to maximize the ELBO for normal data and minimize for sensitive data within the continual learning framework (Kirkpatrick et al. 2017 Shin et al. 2017)
However, this has been experimentally shown to compromise image quality (Heng & Soh 2024
One reason is that the ELBO is unbounded below and can easily diverge to negative infinity as discussed in Section 3.1 To address this issue, for the sensitive data point & and its condition č, Heng & Soh (2024) replace & with completely different data 2, such as noise following uniform distribution. and maximize the ELBO for Z and c. As a result, When the condition č is input, the fine-tuned model generates instead of the sensitive data X. Although this approach effective, it can only be applied to conditional models. Moreover, it cannot work well in PU setting because assumes all unlabeled data are normal. The same limitation applies to other approaches (Brack) et al. 2022 Gandikota et al. 2023 Zhang et al. (2024) as well.
Compared to existing approaches, our objective function reasonably minimizes the ELBO for the sensitive data while preventing divergence to negative infinity, as described in Section 3.1 This enables us to avoid compromising image quality. Furthermore, our approach can be applied to both unconditional and conditional models, as well as PU setting we desire.
4.2 POSITIVE-UNLABLLA LEARNING AND ANOMALY DETECTION
Our approach closely related to PU learning. which trains binary classifier to distinguish between positive and negative data using positive and unlabeled data. Our approach is based on the unbiased PU learning (Du Plessis al. 2014 2015 Kiryo et al. 2017), and has the ideal property that Eq. (88) converges to Eq. (15) as the dataset sizes N,
00.
However, since PU learning is designed for binary classification, it cannot be directly extended to diffusion models. To address this limitation, approach first incorporates binary classification framework into diffusion model training according to Yamanaka et al.] (2019), and then applies PU learning to this framework. A similar approach has been employed in the context of semi- supervised anomaly detection (Takahashi et al., (024).. which aims to train the anomaly detector using the unlabeled and anomaly data. Although this approach has similar problem setting to ours, lacks generative capabilities because it is based on autoencoders (Hinton & Salakhutdinov (0066) To the best of our knowledge, approach is the first to train diffusion models in PU setting.
EXPERIMENTS
5.1 DATA
We used the following image datasets: MNIST (LeCun et al.) 1998), CIFARI0 (Krizhevsky al. 2009). STL10 (Coates et al. 2011), and CelebA (Liu et al. 2015). MNIST is handwritten digits, 10 and STL.0 contain images of animals and vehicles, and CelebA is celebrity faces. We converted MNIST to 32x32 three-channel images, and CelebA to 256x images.
Each dataset was divided into two categories: MNIST into even and odd numbers, CIFARIO and STLIO into animals and vehicles, and CelebA into male and female. We treated one category as normal, and the other sensitive. From these datasets, we prepared training and test data as follows. The training data consist of unlabeled data 24 and labeled sensitive data S, where 2/ include both normal and sensitive data. Meanwhile, the test data contain only normal data. For example, in MNIST, if we treat even numbers as normal, the unlabeled training data i/ include both even and odd numbers, the sensitive training data include only odd numbers, and the test data include only numbers. The number of data points in each dataset is shown in Table
Note that the EI.BO unbounded below, and the negative ELBO (xxo; 0) in Eq. 8 is unhounded ahove.

---

Published as a conference paper at ICLR 2025
Table I: Number of data points in each dataset. The parentheses next the dataset names indicate the normal class.
Image size
U (normal)
U (sensitive)
S
Test
MNIST (even)
32 32
25,000
2,500
2,500
4,926
MNIST (odd)
32
32
25,000
2,500
2,500
5,074
CIFARIO (vehicles)
32 32
20,000
2,000
2,000
4,000
CIFARIO (animals)
32 x 32
20,000
2,000
2,000
6,000
STL10 (vehicles)
96 96
2,000
200
200
3,200
STL10 (animals)
96 x 96
2,000
200
200
4,800
CelebA (male)
256 256
2,000
200
200
7,715
CelebA (female)
256 x 256
2,000
200
200
12,247
5.2 METRIC
We used a custom-defined metric called non-sensitive rate and the Fréchet Inception Distance (FID) score (Heusel et al., 2017) evaluation metrics. The non-sensitive rate is the ratio of the generated samples classified as normal (non-sensitive) by pre-trained classifier. It equals one if all generated samples belong to the normal class and zero if they all belong to the sensitive class. We use this metric to evaluate the frequency with which the diffusion model generates sensitive data. As the classifier, we used convolutional neural network (LeCun al. 1998) for MNIST, and used pre- trained ResNet-34 (He et al., 2016) for the other datasets. We trained or fine-tumed these classifiers on each dataset by solving classification task. The accuracies the test set for the classification task are as follows: MNIST: 98.46%, CIFARIA 10: 95.8%, STL10: 99.5%, and CelebA: 98.45%. The FID is used evaluate the generated samples, with lower score indicating better generative model. We calculated the FID between the samples generated by the diffusion model and the test data. The number generated samples was set be the same as the number of test data points.
5.3 SETUP
We compared our approach with the following methods: the unsupervised method, which minimizes the diffusion model objective f(xo; 0) in Eq. (8) for unlabeled data l/ as in Eq. (9. and the supervised method, which minimizes {(xu: 0) for U/ and maximizes it for sensitive data S as in Eq. (12).
We used the U-Net (Ronneberger et al., 2015) as the noise estimator cg. We performed from-scratch training using MNIST, CIFARIO, and STL10, and fine-tuned pre-trained models using CIFAR10 and CelebA. Our implementations are based on Diffusers (von Platen et al. 2022). For the pre- trained models, used the available models Diffusers for CIFAR1O and CelebA The details of architecture are provided in Appendix.
We optimized these models using AdamW (Loshchilov 2017) and cosine scheduler with warmup. We set the learning rate to 10-4 and set the warmup steps to 500. The batch size was 128 for MNIST and CIFAR 10, 32 for STL10, and 16 for CelebA. The number of epochs was set to 100 for from-scratch training and 20 for fine-tuning, We set the number of steps T during training to 1, 000. For sampling, we used the denoising diffusion probabilistic model (DDPM) scheduler (Ho et al.) 2020) in the from-scratch training, while we used the denoising diffusion implicit model (DDIM) scheduler (Song al. 2020a) in the fine-tuning the pre-trained models, as the pre-trained models large and time-consuming to sample. We set the sampling steps to 1, 000 in the DDPM, and to 0.1.
50 in the DDIM. For the proposed method, we set
We used two machines for the experiments: one with Intel Xeon Platinum 8360Y CPU, 512GB of memory, and NVIDIA A100 SXM4 GPU, and the other with Intel Xeon Gold 6148 CPU, 384GB of memory, and NVIDIA V100 SXM2 GPU. We ran all experiments five times while changing the random seeds.
httros://uugginffccc..co// https://huggingfaae cttee.cooooooeeooeeeeeoo ddpm- ebahq-
far 0-

---

Published as a conference paper at ICLR 2025
Table 2: Comparison of non-sensitive rates for diffusion models with from-scratch training. The parentheses next to the dataset names indicate the normal class.
Unsupervised
Supervised
Proposed
MNIST (even)
0,904=0.003
0,908=0.003
0.978:10.003
MNIST (odd)
0.900=0.006
0.901=0.006
0.975=1.000
CIFARIO 10 (vehicles)
0.702=+0.006
0.760=0.004
0.842+0.006
CIFARIO 10 (animals)
0.917=0.004
0.918=0.004
0.967=10000
STLIO (vehicles)
0.824
0.822-0.008
0.044
STLI0 (animals)
0..08
0.863
0.947
Table 3: Comparison of FID scores for diffusion models with from-scratch training. The parentheses next to the dataset names indicate the normal class.
Unsupervised
Supervised
Proposed
MNIST (even)
3.530
0.130
0.959
MNIST (odd)
3.039
3.056
5.718++0.62
CIFARIO (vehicles)
34.400=+0.622
34.310==0422
32.463-+00.700
CIFARIO 10 (animals)
34.448:00.504
34.613=1.530
39.586:10.573
STL10 (vehicles)
122.79713.5399
122.267:22.11
120.633:14.256
STL10 (animals)
135.999-22.203
135.324=11.099
149.775+11.222
Table 4: Comparison non non-sensitive rates for fine-tuned diffusion models. The parentheses next to the dataset names indicate the normal class.
Pre-trained
Unsupervised
Supervised
Proposed
CIFAR10 (vehicles)
0.008
0.741=40.06
0.763++0.006
1.858-00.00
CIFARIO (animals)
0.681=0.003
0.876=1.0006
0.89210.005
0.954=0.005
CelebA (male)
0.249=0.003
0.866=10.023
0.86810.020
0.973-0.010
CelebA (female)
0.751=0.004
0.905++0.012
0.907+000007
0.942=t0.011
Table 5:
Comparison of FID scores for fine-tuned diffusion models. The parentheses next to the dataset names indicate the normal class.
Pre-trained
Unsupervised
Supervised
Proposed
CIFARIO (vehicles)
84.066-10.403
26.710=10.367
24.45
18.97710.381
CIFAR10 (animals)
35.055-40391
20.313=10.38
19.528-0.277
19.415:0.915
CelebA (male)
100.256:0.261
43.772+2.447
43.465+2.5009
46.877+5.239
CelebA (female)
43.721+0.217
30.520=1.333
30.569
39.131
5.4 RESULTS
5.4.1
COMPARISON OF FID AND
RATE
show the non-sensitive rate and FID for diffusion models with from-scratch training, respectively. Similarly, Tables 4 and 5 show the non-sensitive rate and FID for fine-tuned diffusion models. The parentheses next to the dataset names indicate the normal class. The values before + represent the mean, and the values after + represent the standard deviation. We used bold to highlight the best results and statistically non-different results according to a pair-wise t-test. We used % the p-value. Note that higher non-sensitive rate is better, while smaller FID is better.
Tables and and
8

---

Published as a conference paper at ICLR 2025
(a) Unsupervised.
(b) Supervised.
(c) Proposed.
Figure 2: Generated samples from fine-tuned diffusion models on CIFARIO (vehicles).
(a) Unsupervised.
(b) Supervised.
(c) Proposed.
Figure 3: Generated samples from fine-tuned diffusion models on CelebA (male).
First, we focus on Table 2 Since the ratio of normal data to sensitive data in the unlabeled data LI is 10 : 1, expected that the unsupervised method would generate normal data with approximately 91% probability and sensitive data with 9% probability. As shown in Table 2 the probability of generating normal data ranges from a maximum of 91.7% to a minimum of 76.2% across all datasets. In other words, assuming the classifier is perfect, at least approximately 8.3% of the generated samples are sensitive. While we would expect that supervised method improves the non-sensitive rate, unfortunately, the results remain similar those of the unsupervised method. This indicates that supervised methods do not work well when sensitive data are included in the unlabeled data U. On the other hand, the proposed method shows improvement in non-sensitive rate across all datasets. This demonstrates that the proposed method can reduce the probability of generating sensitive data.
Next, we focus on Table 3 The FID is calculated between the generated samples and the test data, which contain only normal data. It is expected that if the diffusion model generates only normal data, the FID would be improved. On the other hand, learning sensitive data could improve the generation quality simply due the increased amount of data, which might result better FID. As shown in Table 3 there is almost no significant difference in FID between the unsupervised and supervised methods, while the proposed method shows a comparable or slight deterioration in FID. The reason for this will be discussed later. What is important is that the deterioration in FID is extremely small. This indicates that the proposed method can prevent the generation of sensitive data without compromising the image quality.
Finally, we focus on Tables 4 and Note that in the fine-tuning experiments, we also show the non- sensitive rate and FID of the pre-traineds models. CIFAR10 and CelebA datasets imbalanced; CIFAR 10 has more animal images, and CelebA has more female images. As result, the pre- treitrained models show large difference in the probability of generating class over the other. By applying

---

Published as a conference paper at ICLR 2025
42000
Propoud Priteiidd
31000-
Frepond
Anuma
Pacuazzd
Frepeel
01.00
Supaniid
[teaperted)
8.80
Pripaol
21000-
Sqparbool
Umsgeeiael
Unapervied
Unsgervaal Sepesieo
Segirriodd
0.00-
21.00-
0156
0.10
Bils
1030
106
0.30
0:31
0133
0.26
5.00
10.0
0.2
1035
1.00
335
02:00
(a) Rate (vehicles).
(b) FID (vehicles).
(c) Rate (animals).
(d) FID (animals).
Figure 4: Non-sensitive rates and FID scores on CIFAR10 vehicles and animals with various B.
the unsupervised method, we observed an improvement in non-sensitive rate compared to the pre- trained model. As with the from-scratch training, the results of the unsupervised and supervised methods are nearly identical. On the other hand, the proposed method significantly improves non- sensitive rate. In terms of the FID, using the unsupervised, supervised, and proposed methods, we observed improvement compared to the pre-trained model. Compared to from-scratch training, fine-tuning tends to result better FID values with the proposed method. This indicates that the proposed method is well-suited for fine-tuning.
Figures 2 and 3 show the samples by fine-tuned models using the unsupervised. supervised, and proposed methods for CIFARIO (vehicles) and CclebA (male). Since the same random seed was used for both training and generation, the generated images are largely similar. Notably, the images that were sensitive in the unsupervised and supervised methods replaced by normal images in the proposed method. For example, in Figure 2 images of animals are replaced by images of vehicles, and in Figure 3 images of females are replaced by images of males. While the quality of these images is slightly lower, which may explain why the FID of the proposed method is slightly worse than that of other approaches. the overall image quality of the proposed method remains comparable to other methods. Other results, including Stable Diffusion experiments, are provided in Appendix.
5.4.2 SENSITIVITY OF HYPERPARAMETER /3
The proposed method has the hyperparameter 3 defined in Eq. 13 which represents the ratio of sensitive data in the unlabeled data 4. Although this can be estimated by using PU learning approaches (Menon et al. 2015 Ramaswamy et al. 2016 Jain et al., 2016 Christoffel et al. 2016 Nakajima & Sugiyama, 2023). increasing 3 amplifies the loss for sensitive data in Eq. (19). which expected to improve the non-sensitive rate. In this section, we experimentally investigate how adjusting 3 affects the non-sensitive rate and FID.
Figure shows the non-sensitive rate and FID for CIFARIO 10 animals and vehicles when 3 is set to various values. For CIFARIO (vehicles), the best non-sensitive rate and FID are achieved when B is between 0.15 and 0.20. Note that good performance is achieved with 3 value greater than the true value of 1/11 R 0.091. For CIFARIO (animals), the best non-sensitive rate is achieved when == 0.2, while the best FID is achieved for 8 < 0.091. These results indicate that adjusting / can further improve the performance of the proposed method. Analyzing the optimal value of / in the proposed method our important future work.
CONCLUSION
In this paper, we propose positive-unlabeler diffusion models, which prevent diffusion models from generating sensitive data by using unlabeled and sensitive data. With the proposed method, we first incorporate a binary classification framework into diffusion models, and then apply PU learning to this framework. As result, even without labeled normal data, we can maximize the ELBO for normal data and minimize it for labeled sensitive data, ensuring the generation of only normal data. We performed either from-scratch training or fine-tuning of diffusion models on eight different dataset patterns, and confirmed that the proposed method can improve the non-sensitive rate of generated samples without compromising the FID. We also found that adjusting the hyperparameters can further improve both the non-sensitive rate and FID. In the future, we will focus on analyzing hyperparameters and further improving generation quality.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:1 11117988-19999 2021.
Manuel Brack, Patrick Schramowski, Felix Friedrich, Dominik Hintersdorf, and Kristian Kersting. The stable artist: Steering semanties in diffusion latent space. arXiv preprint arXiv:2212.06113, 2022.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wave- grad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009. 00713, 2020.
Marthinus Christoffel, Gang Niu, and Masashi Sugiyama. Class-prior estimation for learning from positive and unlabeled data. In Asian Conference Machine Learning, pp. 221-236. PMLR, 2016.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223, 223. JMLR Workshop and Conference Proceedings, 2011.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In International conference machine learning, pp. 1386-1394. PMLR, 2015.
Marthinus Du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. Advances in neural information processing systems, 27, 2014.
Rohit Gandikota, Joanna Materzynska, Jaden Fiotto Kauto-K and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2426-2436, 2023.
Shansan Gong, Mukai Li, Jiangtao Feng. Zhiyong Wu, and LingPeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.0893,, 2022.
Zayd Hammoudeh and Daniel Lowd. Learning from positive and unlabeled data with arbitrary positive shift. Advances in Neural Information Processing Systems, 33:13088-13099, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp- 770-778, 2016.
Alvin Heng and Harold Soh. Selective amnesia: A continual learning approach forgetting in deep generative models. Advances in Neural Information Processing Systems, 36, 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504-0077 2006.
Jonathan Ho and Tim Salimans. 2rXi1:2207.12598 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-66 6851, 2020.
Shantanu Jain, Martha White, and Predrag Radivojac. Estimating the class prior and posterior from noisy positives and unlabeled data. Advances in neural information processing systems, 29, 2016.
Diederik Kingma. Auto-encoding variational bayes arXiv preprint 13Xiv: 2013.
11

---

Published as a conference paper at ICLR 2025
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526 2017.
Ryuichi Kiryo, Gang Niu, Marthinus C Du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. Advances in neural information processing systems, 30, 2017.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.0977 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applied to document recognition. Proceedings of the IEEE, (1:1))222-2244 1998.
Xiang Li, John Thickstun, Ishaan Guirajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion- Im improves controllable text generation. Advances in Neural Information Processing Systems, 35:4328 -3443,8-444,, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Deep learning face attributes in the wild.
I Loshchilov. Decoupled weight decay regularization. arXiv preprint
2017.
Aditya Menon, Brendan Van Rooyen, Cheng Soon Ong, and Bob Williamson. Learning from cor- rupted binary labels via class-probability estimation. In International conference on machine learning, pp. 125-134. PMLR, 2015.
Yisroel Mirsky and Wenke Lee. The creation and detection of deepfakes: A survey. ACM computing surveys (CSUR), 54(1):1-41, 2021.
Shota Nakajima and Masashi Sugiyama. Positive-unlabele classification under class-prior shift: a prior-invariant approach based on density ratio estimation. Machine Learning, 11(()):89--119 2023.
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad- tts: A diffusion probabilistic model for text-to-speech. In International Conference on Machine Learning, pp. 8599-8608. PMLR, 2021.
Harish Ramaswamy, Clayton Scott, and Ambuj Tewari. Mixture proportion estimation via kernel embeddings of distributions. In International conference on machine learning, pp. 2052-2060.
PMLR, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
Lukas Ruff, Robert A Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus- Robert Müller, and Marius Kloft. Deep semi-supervised anomaly detection. arXiv preprint 19Xiv:1906.02694, 2019.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
Jascha Sohl- Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn ing, pp. 2256-2265. PMLR, 2015.
12

---

Published as a conference paper at ICLR 2025
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456. 2020b.
Hiroshi Takahashi, Tomoharu Iwata, Atsutoshi Kumagai, and Yuuki Yamanaka. Deep positive- unlabeled anomaly detection for contaminated unlabeled data. arXiv preprint arXiv:2405./892, 2024.
Luisa Verdoliva. Media forensics and deepfakes; an overview. IEEE journal of selected topics in signal processing, 14(5):910-932, 2020.
Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Ra- sul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. htttps///iithu.. com/
htwgggingaace/iiffueeeee 2022.
Yuki Yamanaka, Tomoharu Iwata, Hiroshi Takahashi, Masanori Yamada, and Sekitoshi Kanai. Au- toencoding binary classifiers for supervised anomaly detection. In PRICAI 2019: Trends in Ar- rificial Intelligence: 16th Pacific Rim International Conference Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26-30, 2019, Proceedings, Part II 16, 647-659, Springer, 2019.
Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me segeetme--otte Learn- ing to forget in text-to-image diffusion models. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1755-1764, 2024.
13


Reviews:
[{"review_id": "tRrZTrfq1c", "reviewer": "Reviewer_JZxJ", "summary": "This work proposes positive-unlabeled diffusion models that prevent the generation of sensitive data using unlabeled and sensitive data. To address the issue that labeling all sensitive data in the large-scale unlabeled training data is impractical, the authors propose to use a small amount of labeled sensitive data and then propose an approach based on PU learning to train a binary classifier to distinguish between positive and negative data. Simulation results show that the proposed method does effectively prevent generation of sensitive data. Interesting work.", "questions": "I assume that for this method to work, the small amount of labeled sensitive data should well represent the characteristics/properties of all sensitive data. I am just wondering what will happen if there is a mismatch? For example, in the MNIST example that the authors show in the paper, what if the labeled sensitive dataset includes some but not all odd numbers?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "See above summary.", "weaknesses": "I wouldn't count it as a weakness, but more like a further discussion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RPNZcT0iV9", "reviewer": "Reviewer_KfYa", "summary": "This paper presents positive-unlabeled diffusion models to address the challenge of generating sensitive data by diffusion models. Traditional training methods often lead to the inclusion of unwanted sensitive data, prompting the authors to propose a novel approach that utilizes a small amount of labeled sensitive data alongside unlabeled data, which may contain both normal and sensitive instances. The core idea is to maximize the evidence lower bound (ELBO) for normal data using only unlabeled and sensitive data, enabling the generation of only normal content. By treating the unlabeled data distribution as a mixture of normal and sensitive distributions, the approach allows effective model training without needing extensive labeled normal data.\n\nThe overall logic of the paper is coherent, providing a detailed introduction to the relevant technologies and concepts. In particular, the work appears to be well-motivated and the methodology is reasonable. However, the rationale for the proposed method and the challenges associated with it, as well as comparisons with other methods, are not thoroughly explained. For instance, the paper does not clarify why prior methods cannot be directly adapted to diffusion models. The underlying reasons for the unique characteristics of the diffusion model or the challenges inherent in the adaptation process remain unaddressed.", "questions": "1. In Section 4, the disadvantages of previous work and the advantages of the proposed method seem to be inadequately explained and presented. For instance, in Section 4.1, the statement \"Moreover, it cannot work well in the PU setting because it assumes all unlabeled data are normal\" raises the question of what specific issues arise from assuming that all unlabeled data are normal, yet this point is not well articulated.\n2. In Section 4.2, a work related to \"semi-supervised anomaly detection\" is mentioned, which appears to be highly relevant to the proposed method. However, the paper does not clarify why this work cannot be directly adapted to diffusion models. What are the challenges and difficulties associated with this transition?\n3. The experiments involve selecting several classic datasets and categorizing them into two classes. However, the rationale for these datasets being sufficiently representative of sensitive and non-sensitive data is unclear. Although these are binary classification tasks, the boundaries between sensitive and non-sensitive data do not seem to be so distinct. Additionally, in Section 5.4.1 on page 9, it states, \"since the ratio of normal data to sensitive data in the unlabeled data U is 10:1.\" Please clarify the reasoning behind this choice of ratio.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Paper studies an important topic that is highly related to diffusion model and privacy protection.\n- Paper is easy to follow.", "weaknesses": "- The methodological details, and the rationale for adopting the methods, are rather confusing.\n- limited contribution and novelty.\n- The selection of datasets for the experiments and the design of the experimental methods seem to be unreasonable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pO5t1OVBTU", "reviewer": "Reviewer_mThp", "summary": "This paper proposes a novel method to prevent the generation of sensitive images from Diffusion Models. It can estimate the evidence lower bound for normal images without using labeled normal images. Experiments show that the proposed method can alleviate the generation of sensitive images while maintaining high image quality.", "questions": "The questions/suggestions are mainly based on weaknesses.\n- Could authors compare the performance of the proposed method with the works in the related work section? \n- Could authors conduct experiments using SD v1.4 and SD v2.0?\n- It would be better to contain more clear images in the Appendix.\n- Some sentences in the abstract are not very clear. For example, 'Our approach can approximate the evidence lower bound\n(ELBO) for normal (negative) data using only unlabeled and sensitive (positive) data.' Does this mean the sensitive data are unlabeled? But the abstract also says 'we address this problem by using a small amount of labeled sensitive data.'", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper has the following strengths:\n- It is helpful for estimating the evidence lower bound for normal images without relying on labeled examples.\n- The writing of the method part is clear which makes the method easy to understand.", "weaknesses": "This paper has the following weaknesses:\n- The presentation of qualitative examples is not very good. The images in Figure 2 and Figure 3 are small and blurry. Besides, as mentioned by the authors, \"Since the same random seed was used for both training and generation, the generated images are largely similar\". It is harder to see the difference clearly.\n- This paper only tests the performance of the proposed method. It does not have a comparison of previous closely related works.\n- The setting/scenario is not very close to real-world **sensitive** images. For example, it treats even numbers as normal while odd numbers as sensitive. It would be better to consider more real-world common scenarios/settings as in previous works [1, 2].\n- It does not conduct experiments using popular SD such as SD v 1.4, SD v2.0, and SD XL.\n\n--------\n**References**.   \n[1] Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models. In *NeurIPS 2023*.    \n[2] Erasing Concepts from Diffusion Models. In *ICCV 2023*.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xVXCAnvVgp", "reviewer": "Reviewer_o4Dc", "summary": "The paper proposes a new scenario in sensitive generation prevention for DMs, where only an unlabeled dataset and a sensitive dataset are available, rather than a fully labeled training dataset. The authors integrate positive-unlabeled learning methods into DMs, and experimental results demonstrate superior performance across multiple datasets.", "questions": "Question:\n\nSee weakness. Could the authors provide either proof or assumptions that help explain the proposed methods from the perspective of DM's learning process? I believe the results look promising, and with a solid theoretical analysis, I would consider increasing my score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The writing is clear and easy to follow.\n2. The scenario is new and intriguing in the context of DM's sensitive generation prevention.\n3. The experimental results are strong in most cases.", "weaknesses": "The weakness lies in the theory part. There is no guarantee of why the proposed method works, making it seem like positive-unlabeled learning methods are directly applied to DMs. Specifically:\n\nIn Eq(7), the $L_{DM}$ comes from analyzing the lower bound for $\\log p_{\\theta}(x_0)$. However, in Eq(13), there is a linear combination of two distributions, and following Eq(7), it is unclear whether this could be decomposed into terms relating to $l_{BCE}$ for the two given distributions (such as finding a lower bound $\\log (\\beta p_{u}(x) + (1-\\beta) p_{N}(x))$). More insights within the theoretical analysis would help validate whether the proposed positive-unlabeled learning methods can indeed be directly implemented as described in the paper.\n\nBesides, there seems to be a few typos, such as it should be \\citet in L 253.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)