RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Diffusion Model Preliminaries
Denoising Diffusion Probabilistic Model (DDPM) [ 17 ] learns the data distribution \mathbf{x}_{0}\sim q(\mathbf{x}) by reversing the forward noise-adding process.
For the forward process, DDPM defines a Markov process of adding Gaussian noise step by step:

q(\mathbf{x}_{t}|\mathbf{x}_{t-1}):=\mathcal{N}(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I}), (1)

where \beta_{t}\in(0,1) is the hyperparameter controlling the variance.
For the reverse process, DDPM defines a learnable Markov chain starting at p(\mathbf{x}_{T})=\mathcal{N}(\mathbf{x}_{T};\mathbf{0},\mathbf{I}) to generate \mathbf{x}_{0} :

p_{\theta}(\mathbf{x}_{0})=\int_{\mathbf{x}_{1:T}}p(\mathbf{x}_{T})\prod_{t=1}^{T}p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})\ \mathrm{d}\mathbf{x}_{1:T},\qquad p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})=\mathcal{N}(\mathbf{x}_{t-1};\mathbf{\mu}_{\theta}(\mathbf{x}_{t},t),\sigma_{t}^{2}), (2)

where \sigma_{t}^{2} is the untrained time-dependent constant. \theta represents the trainable parameters.
To maximize p_{\theta}(\mathbf{x}_{0}) , DDPM optimizes the Evidence Lower Bound (ELBO) of the log-likelihood [ 17 , 28 ] :

\log p_{\theta}(\mathbf{x}_{0})\geq\mathbb{E}_{q(\mathbf{x}_{1:T}\mid\mathbf{x}_{0})}\left[\log\frac{p_{\theta}(\mathbf{x}_{0:T})}{q(\mathbf{x}_{1:T}\mid\mathbf{x}_{0})}\right]=-\mathbb{E}_{\mathbf{\epsilon},t}\left[||\mathbf{\epsilon}_{\theta}(\mathbf{x}_{t},t)-\epsilon||^{2}\right]+C, (3)

where \epsilon\sim\mathcal{N}(0,\mathcal{I}) , t\sim\text{Uniform}({1,...,T}) and C is a constant. \mathbf{x}_{t} is obtained from Eq. ( 1 ),
and \epsilon_{\theta} is
a function approximator intended to predict the noise \epsilon from \mathbf{x}_{t} . Omitting the untrainable constant in Eq. ( 3 ) and taking its negative yields the loss function of training DDPM.

Conditional diffusion models [ 16 , 34 , 40 ] .
To achieve controllable generation ability, text-to-image diffusion models
incorporate the conditioning mechanism into the model, which are also known as conditional diffusion models, enabling them to learn conditional probability as:

p_{\theta}(\mathbf{x}_{0}|\mathbf{c})=\int_{\mathbf{x}_{1:T}}p(\mathbf{x}_{T})\prod_{t=1}^{T}p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t},\mathbf{c})\ \mathrm{d}\mathbf{x}_{1:T}, (4)

where \mathbf{c} denotes the embedding of condition. For text-to-image synthesis, \mathbf{c}:=\mathcal{T}(\mathbf{y}) , where \mathbf{y} and \mathcal{T} denote the text input and the text encoder, respectively.
Similar to Eq. ( 3 ), through derivation [ 28 ] , we can obtain the ELBO of the conditional log-likelihood:

\log p_{\theta}(\mathbf{x}_{0}|\mathbf{c})\geq-\mathbb{E}_{\mathbf{\epsilon},t}\left[||\mathbf{\epsilon}_{\theta}(\mathbf{x}_{t},t,\mathbf{c})-\epsilon||^{2}\right]+C
Methodology
In this section, we detail the proposed C onditional Li kelihood D iscrepancy ( CLiD ) method. We first introduce the threat model of query-based membership inference in Sec. 3.1 . We then identify the conditional overfitting phenomenon with experimental validation in Sec. 3.2 . We further drive the membership inference indicator based on CLiD in Sec. 3.3 and design two practical membership inference methods in Sec. 3.4 .
We finally provide the implementation details in Sec. 3.5 .
Threat Model
We use the standard security game of membership inference on image-text data following previous work [ 42 , 3 , 4 , 32 ] .
We define a challenger \mathcal{C} and an adversary \mathcal{A} who performs membership inference. \mathcal{C} samples a member set D_{\text{mem}}\leftarrow\mathbb{D} and trains or fine-tunes a text-to-image diffusion model f_{\theta} (i.e., target model) with D_{\text{mem}} . The rest of \mathbb{D} is denoted by hold-out set D_{\text{out}} = \mathbb{D}\setminus D_{\text{mem}} .
For a given data point {(\mathbf{x},\mathbf{c}})\in\mathbb{D} , \mathcal{A} designs an algorithm \mathcal{M} to yield a membership prediction:

\mathcal{M}(\mathbf{x},\mathbf{c},f_{\theta})=\mathbbm{1}\left[\mathcal{M^{\prime}}(\mathbf{x},\mathbf{c},f_{\theta})>\tau\right], (6)

where \mathcal{M^{\prime}} denotes an indicator function that reflects membership information, and \tau denotes a tunable decision threshold of query-based membership inference [ 32 , 11 , 23 , 13 ] .

We consider a grey-box setting consistent with previous query-based methods [ 32 , 11 , 23 , 13 ] . This setting assumes that \mathcal{A} has access to the intermediate outputs of models without knowledge of specific model parameters.
For the given image-text data point (\mathbf{x},\mathbf{c}) , we assume that \mathbf{x} and \mathbf{c} always correspond within the dataset \mathbb{D} . This assumption is evident in scenarios where dataset copyright owners perform membership inference to audit usage.

Conversely, challenger \mathcal{C} can mitigate the effectiveness of membership inference during training by utilizing data augmentation or even adaptive defense methods, which we discuss in Sec. 4.5 . Our work primarily focuses on the fine-tuning scenarios of text-to-image diffusion models because the weights of pre-trained models are readily available, making this scenario more prone to copyright and privacy risks [ 50 , 35 ] .
And we also conduct experiments on pre-trained text-to-image diffusion models ( Sec. 4.6 ) to demonstrate the effectiveness of our method even in pre-training scenarios.
Condition Likelihood Discrepancy
In this section, we derive a membership inference indicator for a given individual sample based on Assumption 3.1 .
Calculating FID requires sampling lots of images from the p distribution,
which is impractical under membership inference scenarios.
Instead, we employ Kullback-Leibler (KL) divergence as the distance metric, which is widely used and computationally convenient
(the usage of other metrics is discussed in Appendix C ). Then we have the following theorem:

(Proof in the Appendix B )
When using D=D_{KL} as distance metric, Assumption 3.1 is equivalent to:

\mathbb{E}_{q_{mem}(\mathbf{x},\mathbf{c})}[\log p(\mathbf{x}|\mathbf{c})-\log p(\mathbf{x})]\geq\mathbb{E}_{q_{out}(\mathbf{x},\mathbf{c})}[\log p(\mathbf{x}|\mathbf{c})-\log p(\mathbf{x})]+\delta_{H}, (9)

where

\delta_{H}=H(q_{out}(\mathbf{x}))+\mathbb{E}_{\mathbf{c}}[H(q_{mem}(\mathbf{x}|\mathbf{c}))]-H(q_{mem}(\mathbf{x}))-\mathbb{E}_{\mathbf{c}}[H(q_{out}(\mathbf{x}|\mathbf{c}))]

Let us define:

\mathbb{I}(\mathbf{x},\mathbf{c})=\log p(\mathbf{x}|\mathbf{c})-\log p(\mathbf{x})

If \delta_{H} is negligible, then according to Eq. 9 , it holds that \mathbb{E}_{q_{\text{mem}}}[\mathbb{I}(\mathbf{x})]\geq\tau\geq\mathbb{E}_{q_{\text{out}}}[\mathbb{I}(\mathbf{x})] , where \tau is a constant intermediate between the left-hand side and right-hand side.
Membership inference is then posed as follows: given an input instance (\mathbf{x},\mathbf{c}) , measuring \mathbb{I}(\mathbf{x},\mathbf{c}) to predict how probable it is that the input is a sample from q_{\text{mem}} rather than q_{\text{out}} .
Intuitively, if \mathbb{I}(\mathbf{x}) exceeds a threshold \tau , the instance is likely from q_{\text{mem}} ; otherwise, it belongs to q_{\text{out}} . In the community of membership inference methods [ 56 , 5 , 3 , 4 , 32 , 11 , 23 , 13 ] , setting such a threshold \tau is a standard practice to differentiate between the two distributions.
Therefore, we can utilize the indicator \mathbb{I}(\mathbf{x},\mathbf{c}) for membership inference.
Since Eq. ( 11 ) actually involves measuring the likelihood discrepancy under different conditions of diffusion models, we call it C onditional Li kelihood D iscrepancy ( CLiD ).

In order to calculate the likelihoods in Eq. ( 11 ) for a given data point (\mathbf{x},\mathbf{c}) , we utilize the ELBOs in Eq. ( 5 ) and Eq. ( 3 ) as an approximation of the log-likelihoods:

\mathbb{I}(\mathbf{x},\mathbf{c})=\mathbb{E}_{t,\mathbf{\epsilon}}\left[||\mathbf{\epsilon}_{\theta}(\mathbf{x}_{t},t,\mathbf{c}_{null})-\epsilon||^{2}\right]-\mathbb{E}_{t,\mathbf{\epsilon}}\left[||\mathbf{\epsilon}_{\theta}(\mathbf{x}_{t},t,\mathbf{c})-\epsilon||^{2}\right], (12)

where \mathbf{c}_{\text{null}} denotes an empty text condition input used to estimate the approximation of \log p_{\theta}(\mathbf{x}) .
Practical Considerations
Reducing conditions to obtain \mathbf{c}^{*} . We consider three methods for diverse reduction:
(1) Simply taking the first, middle, and last thirds of the sentences as text inputs. (2) Randomly adding Gaussian noises with various scales to the text embeddings. (3) Calculating the importance of words in the text [ 49 , 51 ] and replacing them with “pad” tokens by varying proportions in descending order. For all three methods, we additionally use the null text input as \mathbf{c}^{*}_{k} . These methods are all effective and we use (3) with k=4 in subsequent experiments (details in Appendix D ).

Monte Carlo sampling. Let M and N denote the Monte Carlo sampling numbers of estimating \mathcal{L}(\mathbf{x},\mathbf{c}) and \mathcal{D}_{\mathbf{x},\mathbf{c},\mathbf{c}^{*}_{i}} , respectively.
We set M=N to achieve result reuse between Eq. ( 15 ) and Eq. ( 13 ), reducing the number of Monte Carlo sampling.
Hence the overall query count of one data point is M+K\cdot N . Significant effects can be observed even when M,N=1 (Fig. 3 ).

Classifiers of \text{CLiD}_{vec} . Due to the simplicity of the feature vectors, we do not need a neural network as the classifier [ 11 ] . Simpler classifiers help to prevent overfitting.
In our experiments, we utilize XGBoost [ 6 ] and utilize its predict confidence.
Main Results
† When conducting SecMI [ 11 ] , we observe that the thresholds obtained on the shadow model sometimes do not transfer well to the target model.

Over-training scenario. In Tab. 1 , models are trained for excessive steps on all three datasets, resulting in significant overfitting. We observe that under this over-training scenario, both of our methods nearly achieve ideal binary classification effectiveness. For instance, \text{CLiD}_{th} achieves over 99% ASR, AUC and TPR@1%FPR value on the MS-COCO dataset [ 29 ] .
With this training setup, the metrics for different baselines are very similar.
Even the simplest loss-based method [ 32 ] (with the query count of 1) also yields satisfactory results compared with other high query count methods.
Therefore, we emphasize that such unrealistic over-training scenario fail to adequately reflect the effectiveness differences among various membership inference methods.

Real-world training scenario. In Tab. 2 , we adjust the training steps simulating real-word training scenario [ 19 ] and utilize default data augmentation [ 20 ] . The best value of ASR and AUC of baseline methods decreases to around 65%, and the best value of TPR@1%FPR decreases to around 5%, indicating insufficient effectiveness of previous member inference methods in real-world training scenarios of text-to-image diffusion models.
In contrast, our methods maintain ASR above 86% and AUC above 93%, exceeding the best baseline values by about 30%. The TPR@1%FPR of our methods exceeds the best baseline values by 50%~60%.
The results demonstrate the effectiveness of our methods across various data distributions and scales in real-world training scenarios.
Performance on Various Training Steps
From Tab. 1 and Tab. 2 , we find that the training steps greatly influence the effectiveness of membership inference.
All membership inference methods tend to exhibit satisfactory performance when the model is trained for an excessive number of steps that conflicts with real-world scenarios.
Therefore, we emphasize that the effectiveness trajectory of membership inference across varying training steps should also be utilized to evaluate different methods. Better membership inference methods should reveal membership information earlier as training progresses.

In Fig. 2 , we fine-tune Stable Diffusion models with MS-COCO dataset for varying training steps under real-world training setting and report the AUC values of different membership inference methods.
It can be observed that as the training progresses, \text{CLiD}_{th} exhibits a significantly faster increase in effectiveness trajectory. By 25,000 steps, \text{CLiD}_{th} effectively exposes membership information, whereas other baselines achieve similar results only at around 150,000 steps. This demonstrates that our method can effectively reveal membership information when the overfitting degree of the text-to-image diffusion model is weaker.
Ablation Study
To conduct ablation study, we vary the Monte Carlo sampling count in Eq. ( 14 ) and Eq. ( 15 ), perform \text{CLiD}_{th} with MS-COCO dataset under real-world training setting and report the AUC values in Fig. 3 .
To further compare the effects of Eq. ( 14 ) and Eq. ( 15 ), we discard each term in Eq. ( 16 ) and denote it as M/N=0 .
We also include the result of the best baseline, SecMI [ 11 ] , as a comparison.

Effect of \mathcal{D}_{\mathbf{x},\mathbf{c},\mathbf{c}^{*}} . In Fig. 3 , results of "M=1, N=0" and "M=1, N=1" show a significant improvement of membership inference by including \mathcal{D}_{\mathbf{x},\mathbf{c},\mathbf{c}^{*}} . Results of "M=5, N=0" and "M=1, N=1" further show that the method utilizing \mathcal{D}_{\mathbf{x},\mathbf{c},\mathbf{c}^{*}} performs much better under the same sampling numbers. Additionally, the results of "M=0, N=1" and "M=1, N=1" indicates that only considering both Eq. ( 14 )
and Eq. ( 15 ) achieves the optimal performance.

Monte Carlo sampling numbers. In Fig. 3 , we observe that when setting M=N , the performance improves as the number of Monte Carlo sampling increases. And the performance is improved slightly when M,N>3 . Hence, we set M,N=3 to ensure the balance between a low query count and satisfied performance. Moreover, the results of "M=1, N=1" and "SecMI" also demonstrate that \text{CLiD}_{th} outperforms previous works even with a fewer query count.
Resistance to Defense
Since data augmentation is commonly used in training and can mitigate the effectiveness of membership inference [ 11 ] , we use it to evaluate the performance
of methods under defense.
As the baseline methods already exhibit weak performance under real-world training setting, we opt not to incorporate additional data augmentation. Instead, we remove the default data augmentation from training scripts [ 20 ] to observe the effectiveness change of different methods. We fine-tune Stable Diffusion models for 50,000 steps with MS-COCO, report the metrics, and calculate the metrics changes in Tab. 3 .
We observe that the effectiveness of all membership inference methods declines after data augmentation is introduced during training.
Note that PFAMI [ 13 ] exhibits the highest sensitivity to data augmentation since it infers membership by probability fluctuation after images are perturbed, which also explains its significant performance decline between Tab. 1 and Tab. 2 .
Compared to the baselines, our method exhibits the smallest decrease, which indicates its stronger resistance to data augmentation.

Adaptive defense. We further consider adaptive defense: assuming the trainers are aware of our methods and perturb the text of image-text datasets before training. We consider the following adaptive defense methods: (1) rephrasing the original text 3 3 3 We utilize ChatGPT-3.5 with the following prompt: ”Please rewrite the following sentences while keeping the key semantics.” , (2) randomly deleting 10%, 30% words in text, and (3) shuffling 50% of the image-text mappings in the dataset.
In Tab. 5 , we observe that except for shuffling , the other adaptive defense methods have almost no effect on \text{CLiD}_{th} . And shuffling damages the model utility (too high FID values), rendering this defense meaningless.

† We omit the discussion of PIA as it shows no effectiveness at this training steps, with the metrics consistently approximating random guessing.

‡ The TPR@1%FPR value changes significantly here because its ROC curve is very sharp when FPR close to 0 .

† Compared to other methods, the increase in FID caused by shuffling is unacceptable for generative models.
Performance on Pre-trained Models
In this part, we evaluate the effectiveness of membership inference methods for the pre-training setting.
To ensure the distribution consistency between the member and hold-out set, we respectively select 2500 samples from the LAION-Aesthetics v2 5+ and LAION-2B MultiTranslated [ 45 ] as member/hold-out set following [ 12 ] , and conduct membership inference on Stable Diffusion v1-5 [ 40 ] .
As shown in Tab. 5 , our method consistently outperforms the baselines across all three metrics.
Related Works
Copyright protection for image-text datasets. To protect the copyright of image-text datasets, some works [ 44 , 58 , 46 ] incorporate imperceptible perturbations to render the released datasets unusable.
Besides, other works [ 50 , 9 ] utilize backdoor or watermarking to track the usage of image-text datasets.
In contrast, our method demonstrates the possibility of auditing the unauthorized usage of individual image-text data points utilizing membership inference.

Membership inference on diffusion models. Recently, several membership inference methods on diffusion models have been proposed. In the grey-box or white-box setting, Carlini et al. [ 4 ] firstly conduct membership inference on unconditional diffusion models by conducting LiRA (Likelihood Ratio Attack) [ 3 ] , with the requirement of training multiple shadow models. Matsumoto et al. [ 32 ] make the first step by utilizing diffusion loss to conduct query-based membership inference. Some works [ 11 , 23 ] leverage the DDIM [ 48 ] deterministic forward process [ 22 ] to access the posterior estimation errors of diffusion models. And Fu et al. [ 13 ] leverage the probability fluctuations by perturbing image samples. Few works consider the black-box settings [ 54 , 35 ] . However, these studies either assume partial knowledge of member set data [ 54 ] or assume extensive fine-tuning steps ( 100\sim 500 epochs) [ 35 ] , both of which do not align with real-world scenarios.
Conclusion
In this paper, we identify the phenomenon of conditional overfitting in text-to-image models and propose CLiD-MI , the membership inference framework on text-to-image diffusion models utilizing the derived indicator, conditional likelihood discrepancy. Experimental results demonstrate the superiority of our method across various data distributions and scales, while also showing the resistance against early stopping and data augmentation strategies. Our method aims to inspire a new direction for the community regarding unauthorized usage auditing of text-image data.
D.1 Monte Carlo Sampling
In our method, the key to accurate membership inference lies in estimating ELBO with fewer sampling steps for better precision.
To achieve this, firstly, we reduce the number of Monte Carlo samples by directly estimating the ELBO difference (Eq. ( 13 )). Secondly, recalling Monte Carlo sampling using (t_{i},\epsilon_{i}) pairs with \epsilon_{i}\sim\mathcal{N}(0,\mathbf{I}) and t_{i}\sim[1,1000] , we explore the effect of the sampling time t_{i} .
We conduct a single Monte Carlo sampling test using MS-COCO on real-word training setting and report the AUC values in Fig. D.1 .

In Fig. D.1 , we observe that the single Monte Carlo estimation achieves optimal accuracy when t_{i}\in[400,500] . Similar results are shown in [ 28 ] . Therefore, consistent with [ 28 ] , we sample at intervals of 10 centered around the timestep 450 .
For example, if the estimation number (i.e., M,N in Eq ( 14 )) is 3 , we use the time list of [440,450,460] .
Note that [ 4 , 11 ] indicate that for DDPM of Cifar-10 [ 25 ] , the best estimation timestep is around 100 .
This difference may arise from the different signal-to-noise ratios of images with various resolution [ 18 ] .
This finding suggests that the Monte Carlo sampling timestep should be designed differently for diffusion models of different scales.
D.2 Reduction Methods
In implementation, we actually diversely reduce the condition \mathbf{c} to \mathbf{c}^{*} and calculate p_{\theta}(\mathbf{x}|\mathbf{c}^{*}) to approximate p_{\theta}(\mathbf{x}) .
In this part, we evaluate the effectiveness of different reduction methods. We consider three methods in Sec. 3.5 : (1) Simply Clipping. We simply use the first, middle, and last thirds of the sentences as text inputs. (2) Gaussian Noise. We add Gaussian noises with the scales of 50\%,70\%,90\% to the overall text embeddings. (3) Importance Clipping. We calculate the importance of words in the text 4 4 4 https://github.com/ma-labo/PromptCharm [ 49 , 51 ] and replace them with “pad” tokens in descending order by varying proportions of 30\%,50\%,70\% . For all three methods, we additionally use the null text as a \mathbf{c}^{*} . The experiments are conducted on the real-world training setting with MS-COCO dataset. And we also employ null text solely to compute Eq. ( 14 ) without reduction methods for comparison.

In Tab. D.1 , we observe that Importance Clipping achieves the best results due to its more deterministic reduction. Additionally, we note that all three reduction methods exhibit satisfactory results, demonstrating the general applicability of our method. Comparing the results without the usage of reduction methods, the results validate the effectiveness of reduction methods in Sec. 3.5 .
Figure 1: FID values and the FID differences
of synthetic images ( 2500/2500 samples for member/hold-out set) under different conditions of member set and hold-out set.; FID values and the FID differences
of synthetic images ( 2500/2500 samples for member/hold-out set) under different conditions of member set and hold-out set.
Figure A.1: Metric values and the metric differences of synthetic images, with the same setting as Sec. 3.2 .; Figure A.1: Metric values and the metric differences of synthetic images, with the same setting as Sec. 3.2 .
Figure D.1: Effectiveness of single Monte Carlo estimation of various timesteps. Small t_{i} corresponds to less noise added, and large t_{i} corresponds to significant noise. AUC value is highest when the timestep is around 450 .; Figure D.1: Effectiveness of single Monte Carlo estimation of various timesteps. Small t_{i} corresponds to less noise added, and large t_{i} corresponds to significant noise. AUC value is highest when the timestep is around 450 .
Figure 2: Effectiveness trajectory on various training steps.; Effectiveness trajectory on various training steps.
Figure 3: Performance of \text{CLiD}_{th} under various Monte Carlo sampling numbers (i.e., query count). The legend labels are sorted in ascending order of AUC values and SecMI. M/N=0 indicates that the corresponding term is not utilized in Eq. ( 16 ).; Performance of \text{CLiD}_{th} under various Monte Carlo sampling numbers (i.e., query count). The legend labels are sorted in ascending order of AUC values and SecMI. M/N=0 indicates that the corresponding term is not utilized in Eq. ( 16 ).
Results under Over-training setting. We mark the best and second-best results for each metric in bold and underline , respectively. Additionally, the best results from baselines are marked in blue for comparison.
Results under Real-world training setting. We also highlight key results according to Tab. 1 .
The performance of different methods under no augmentation and default augmentation.
Effectiveness of \text{CLiD}_{th} in adaptive defense. We calculate the FID [ 15 ] with 10,000 unseen MS-COCO samples to access the model utility.
The membership inference performance with different reduction methods. "Null" denotes employing null text solely to compute Eq. ( 14 ) without reduction methods.