Introduction
Text-to-image diffusion models have achieved remarkable success in the guided generation of diverse, high-quality images based on text prompts,
such as Stable Diffusion [ 40 , 36 ] , DALLE-2 [ 37 ] , Imagen [ 43 ] , and DeepFloyd-IF [ 26 ] .
These models are increasingly adopted by users to create photorealistic images that align with desired semantics.
Moreover, they can generate images of specific concepts [ 27 ] or styles [ 53 ] when fine-tuned on relevant datasets.
However, the impressive generative capabilities of these models depend heavily on high-quality image-text datasets, which involve collecting image-text data from the web.
This practice raises significant privacy and copyright concerns in the community.
The pre-training and fine-tuning processes of text-to-image diffusion models can cause copyright infringement, as they utilize unauthorized datasets published by human artists or stock-image websites [ 2 , 7 , 52 , 39 , 38 ] .

Membership inference (also known as the membership inference attack) is widely used for auditing privacy leakage of training data [ 47 , 3 ] , defined as determining whether a given data point has been used to train the target model.
Dataset owners can thus leverage membership inference to determine if their data is being used without authorization [ 33 , 10 ] .

Previous works [ 4 , 11 , 23 , 13 , 12 , 32 ] have attempted membership inference on diffusion models. Carlini et al. [ 4 ] employ LiRA (Likelihood Ratio Attack) [ 3 ] to perform membership inference on diffusion models.
LiRA requires training multiple shadow models to estimate the likelihood ratios of a data point from different models, which incurs high training overhead (e.g., 16 shadow models for DDPM [ 17 ] on CIFAR-10 [ 25 ] ), making it neither scalable nor applicable to text-to-image diffusion models. Other query-based membership inference methods [ 32 , 11 , 23 , 13 ] design and compute indicators to evaluate whether a given data point belongs to the member set. These methods require only a few or even a single shadow model, making them scalable to larger text-to-image diffusion models.
However, these methods mainly estimate model memorization for data points and do not fully utilize the conditional distribution of image-text pairs.
Consequently, they achieve limited success only under excessively high training iterations and fail under real-world training scenarios with common data augmentation methods (Tab. 2 ), which does not reflect real training scenarios.
Text-to-image diffusion models have been demonstrated excellent performance in zero-shot image generation [ 40 , 36 , 1 ] , indicating their strong generalization, which makes it difficult to distinguish membership by directly measuring overfitting to data points.
And due to the stochasticity of diffusion training loss [ 17 , 40 ] , this kind of measuring becomes more challenging.

To address the challenges, we first identify a Conditional Overfitting phenomenon of text-to-image diffusion models with empirical validation, where the models exhibit more significant overfitting to the conditional distribution of the images given the corresponding text prompts than the marginal distribution of the images only. It inspires the revealing of membership by leveraging the overfitting difference.
Based on it, we propose to perform membership inference on text-to-image diffusion models via C onditional Li kelihood D iscrepancy ( CLiD ).
Specifically, CLiD quantifies overfitting difference analytically by utilizing Kullback-Leibler (KL) divergence as the distance metric and derives a membership inference indicator that estimates the discrepancy between conditional likelihood of image-text pairs and likelihood of images.
We approximate the likelihoods by employing Monte Carlo sampling
on their ELBOs (Evidence Lower Bounds),
and design two membership inference methods: a threshold-based method \text{CLiD}_{th} and a feature vector-based method \text{CLiD}_{vec} .

We conduct extensive experiments on three text-to-image datasets [ 29 , 57 , 27 ] with different distributions and sizes using the mainstream open-sourced text-to-image diffusion models [ 8 , 41 ] under both fine-tuning and pre-training settings.
First, our methods consistently outperform existing baselines across various data distributions and training scenarios. For example, \text{CLiD}_{vec} achieves average improvements over the best baseline by 28.7%, 29.8%, and 55.9% in terms of ASR, AUC, and TPR@1%FPR metrics, respectively.
Second, we suggest evaluating the membership inference methods by observing the effectiveness trajectory with increasing training steps of a model. The experimental results show our methods reveal membership earlier with fewer training steps.
Next, ablation studies further demonstrate the effect of our CLiD indicator, even with fewer query count, our method still outperforms baseline methods ( Fig. 3 ).
Last, experiments show that our methods exhibit stronger resistance to data augmentation, and exhibit resistance to even adaptive defenses.