Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Authors: Xiaoyu Wu
Jiaru Zhang
Steven Wu
Venue: ICML 2025 poster (2025)
Keywords: Data Extraction
Copyright Protection
Privacy and Security
Diffusion Models
Trustworthy AI
Primary area: social_aspects->privacy
TL;DR: Extrapolated guidance from pretrained to fine-tuned DMs enables strong fine-tuning data extraction.

Abstract: Diffusion Models (DMs) have evolved into advanced image generation tools, especially for few-shot fine-tuning where a pretrained DM is fine-tuned on a small set of images to capture specific styles or objects. Many people upload these personalized checkpoints online, fostering communities such as Civitai and HuggingFace. However, model owners may overlook the potential risks of data leakage by releasing their fine-tuned checkpoints. Moreover, concerns regarding copyright violations arise when unauthorized data is used during fine-tuning.  In this paper, we ask: "Can training data be extracted from these fine-tuned DMs shared online?" A successful extraction would present not only data leakage threats but also offer tangible evidence of copyright infringement. To answer this, we propose FineXtract, a framework for extracting fine-tuning data.  Our method approximates fine-tuning as a gradual shift in the model's learned distribution---from the original pretrained DM toward the fine-tuning data. By extrapolating the models before and after fine-tuning, we guide the generation toward high-probability regions within the fine-tuned data distribution. We then apply a clustering algorithm to extract the most probable images from those generated using this extrapolated guidance. Experiments on DMs fine-tuned with datasets such as WikiArt, DreamBooth, and real-world checkpoints posted online validate the effectiveness of our method, extracting approximately 20% of fine-tuning data in most cases, significantly surpassing baseline performance. The code is available.

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Xiaoyu Wu" Jiaru Zhang² Zhiwei Steven Wu
Abstract
1. Introduction
Diffusion Models (DMs) have become power- ful image generation tools, especially for few- shot fine-tuning where a pretrained DM is fine- tuned on a small image set to capture specific styles or objects. Many people upload these per- sonalized checkpoints online, fostering commu- nities such Civitai and HuggingFace. How- ever, model owners may overlook the data leak- age risks when releasing fine-tuned checkpoints. Moreover, concerns regarding copyright viola- tions arise when unauthorized data is used during fine-tuning. In this paper, we ask: "Can train- ing data be extracted from these fine-tuned DMs shared online?" A successful extraction would present not only data leakage threats but also of- fer tangible evidence of copyright infringement. To answer this, we propose FineXtract, frame- work for extracting fine-tuning data. Our method approximates fine-tuning as a gradual shift in the model's learned distribution- ffrmm the original pretrained DM toward the fine-tuning data. By extrapolating the models before and after fine- tuning, we guide the generation toward high- probability regions within the fine-tuned data dis- tribution. We then apply a clustering algorithm to extract the most probable images from those gen- erated using this extrapolated guidance. Experi- ments on DMs fine-tuned with datasets includ- ing WikiArt, DreamBooth, and real- woold check- points posted online validate the effectiveness of our method, extracting about 20% of fine-tuning data in most cases. The code is available
Recent years have witnessed the advancement of Diffusion Models (DMs) in computer vision. These models demon- strate exceptional capabilities across various tasks, includ- ing image editing (Kawar et al.] 2022). and video edit- ing (Yang et al.) (022). among others. Particularly note- worthy is the advent of few-shot fine-tuning methods (Hu) et al. 2021 Ruiz et al. 2023 Qiu et al., 2023), in which pretrained model fine-tuned to personalize generation based on a small set of training images. These approaches have significantly reduced both memory and time costs in training. Moreover, these techniques offer powerful tools for adaptively generating images based on specific subjects or objects, embodying personalized Al and making Al ac- cessible to everyone.
Building on these innovations, several communities, such as Civitai (civ) and HuggingFace (hug). have emerged, hosting tens of thousands of fine-tuned checkpoints and at- tracting millions of downloads. Although many users will- ingly share their fine-tuned models, they may be unaware of the risk of data leakage inherent in this process. This is particularly concerning when fine-tuning involves sensi- tive data, such as medical images, human faces, or copy- righted material. Moreover, many of these checkpoints are fine-tuned using unauthorized data, including artists' work. This unauthorized fine-tuning process raises signif- icant concerns regarding "reputational damage, economic loss, plagiarism and copyright infringement" as mentioned in Tiang et al. (2023), and has prompted numerous objec- tions from data owners (Liang et al., 2023 Wu et al. 2024 Shan et al. 2023).
In this paper, we pose critical question: "Is it possi- ble to extract fine-tuning data from these fine-tuned DM checkpoints released online?" Successfully doing so would confirm that fine-tuning data is indeed leaked within these checkpoints. Moreover, the extracted images could serve as strong evidence that specific data was used in the fine-tuning process, thereby aiding those whose rights have been infringed to seek legal protection and take necessary legal action.
'Work done during internship at CMU. 'Camegic Mellon University, Piusburgh, PA 15213, USA 'Purdue University, West Lafayeue. IN 47907. USA. Correspondence to: Xi- aoyu Wu <nicholaswu2022e gmmal.comm....m Zhiwei Steven Wu latevenwu@cmu.edde
Proceedings of the 42nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
https: /ttp//gittti http//tttubu..o// com www.chollooooooo cho.aall2222
More specifically, extracting fine-tuning data can be seen
FineXtract

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Figure 1: Extraction results from real-world fine-tuned checkpoints on HuggingFace using our FineXtract. Top: Extracted images. Bottom: Corresponding training images.
as targeting specific portions of the training data, whereas previous work on extracting data from diffusion models has mainly focused general generative processes (Car- lini et al. 2023 Somepalli et al.] 2023a b), often overlook- ing more detailed or interesting data. To address this gap, we propose new framework, called FineXtract, for ef- ficiently and accurately extracting training data from the extrapolated guidance between DMs before and after fine- tuning. We begin by providing parametric approximation of the fine-tuned DMs distribution, modeling it as inter- polation between the pretrained DMs' distribution and the fine-tuned data distribution. With this approximation, we demonstrate that extrapolating the score functions of the pretrained and fine-tuned DMs can effectively guide the de- fine-
We approximate the learned distribution during the fine-tuning process of DMs and demonstrate how this guides the model towards the high-density regions of the fine-tuned data distribution.
We propose new framework, FineXtract, for ex- tracting fine-tuning datasets using this approximation. With clustering algorithm, our method can extract images visually close to fine-tuning dataset.
Experimental results on fine-tuned checkpoints on various datasets (WikiArt, DreamBooth), various DMs and real-world checkpoints from HuggingFace validate the effectiveness of our methods.
noising process toward the high-density regions tuned data distribution, process we refer as model guid- ance. We then generate set of images within such high- density regions and apply clustering algorithm to identify the images that are most likely to match the training data within the fine-tuning dataset.
2. Background and Related Works
2.1. Diffusion Models and Few-shot Fine-tuning
Diffusion Models and Score Matching. Diffusion Mod- els (DMs) (Ho et al. 2020) Sohl-Dickstein et al. 2015) are generative models that approximate data distributions by gradually demoising variable initially sampled from Gaussian distribution. These models consist of forward diffusion process and backward denoising process. In the forward process, noise NN 1) is progressively added to the input image over time t. following the equation Tt Varao + VT CyE. Conversely, the backward process, DMs aim to estimate and remove the noise using a noise-prediction module, es, from the noisy image æ.. The difference between the actual and predicted noise forms the basis of the training loss, known the diffusion loss, which is defined LDm Ee~A((0,),, [Ilea . () -=ll²]-
Our method can be applied to both unconditional and con- ditional DMs. Specifically, when the training caption is available, we approximate the learned distribution DMs conditional caption unconditional DM learned distribution and the conditional data distribution. Combined with model guidance, this leads to an extrapolation from the noise predicted by the pretrained unconditional DM to that by the fine-tuned con- ditional DM, guiding generation toward the high-density region of the fine-tuned data distribution conditioned on c. Experiments across different datasets, DM structures, and real-world checkpoints from HuggingFace demonstrate the effectiveness of our method, which extracts around 20% of images in most cases (See Fig. I for visual examples).
interpolation between the
Another series of works focus on score matching, offer- ing insights into DMs from a different perspective (Vin-) cent 2011 Song Ermon 2019 Song et al., 2020). Score matching aims learn score network s0(xr) trained to
In summary, our contributions are as follows:
2

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
predict the score (i.e., the gradient of the log probability function) V. g.log((2) of data x within real data distribu- tion gax)) (Vincent 2011). To improve accuracy and sta- bility, subsequent research proposes predicting the score of the Gaassiann-ertuurkie
3. Threat Model and Metries
3.1. Threat Model
Our threat model involves extracting training data from a fine-tuned DM alongside its corresponding pretrained DM, with two key parties: model providers and attackers.
data distribution g( (Song & Er- mon 2019: Song et al. 2020h: so t) s Va, log q(xx)) =
where
I Og These works show
v1 mr
Model providers. Providers fine-tune a pretrained model 0 using an image dataset Xo. After fine-tuning, they upload the fine-tuned model checkpoint 0' to specific websites, in- cluding necessary details such as the name of the pretrained model 0 to make the fine-tuned model checkpoint usable. Additional training details, such as training captions, are sometimes provided (civ) hug).
strong alignment between the predicted noise ca( 1) and the score lolog g(z).
Few-shot Fine-tuning. Few-shot fine-tuning in DMs (Gal
et al. 2022 Hu et al. 2021 Ruiz et al. 2023) aims to per- sonalize these models using a limited set of images, en- abling the generation of customized content. Gal et al.
(022) introduced a technique that incorporates new to- kens within the embedding space of a frozen text-to-image model to capture the concepts in the provided images. However, this method has limitations in accurately re- producing the detailed features of the input images (Ruiz et al. 2023). To address this, Ruiz et al. (2023) proposed DreamBooth which fine-tunes most parameters in DMs using a reconstruction loss to capture details and a class- specific preservation loss to ensure alignment with textual prompts. Additionally,HHuu et al. (202T) introduced LoRA, lightweight fine-tuning approach that inserts low-rank lay- ers to be trained while keeping other parameters frozen.
A Attackers Attackers download the checkpoints O' from these websites. They also acquire the pretrained model 0. By default, we assume that the attacker can access the training caption, following previous work (Somepalli et al. 2023a Carlini et al. 2023). (This a reasonable assump- tion as more discussed in Appendix Sec. A where we show that even without direct access, captions can be partially extracted using inversion on linear projection layers.) At- tackers have no prior knowledge about the image dataset Xo. Their goal is to extract as much information about Xo as possible. A successful attack occurs when attacker ex- tracts an image set X that is almost identical to the training image set Xp-
2.2. Memorization and Data Extraction in Diffusion Models
3.2. Evaluation Metries
Some studies have highlighted DMs' tendencies toward data memorization and methods have been proposed to ex- tract training data based on it. Carlini et al. (2023) used a graph algorithm to identify the generated data most likely to have been included in the training set, thereby retriev- ing DM's memorized training data. Further investigations (Somepalli et al., 2023a b) explore the underlying causes of this memorization, revealing that conditioning plays significant role, and the nature of training prompts notably influences the likelihood of reproducing training samples. However, these studies are primarily empirical with no parametric formulation on learned distribution of DMs, and they do not address personalization scenarios. In contrast, our approach introduces a parametric approximation of the learned distribution of fine-tuned DM,, enabling the design of pipeline that efficiently extracts training samples from online fine-tuned checkpoints.
Attacker produces an extracted dataset X, which is eval- uated by comparing it with the training image set Xo. Specifically, we consider the following two metrics:
Metric 1: Average Similarity (AS). Average similarity is computed between images in the extracted dataset X and those in the training dataset Xo. The metric is defined as:
X²l
AS Xo, 8) = X Σ
max sim x6", (01).
(1)
Here, sim(,)) denotes the similarity function, with output ranging from 0 to 1. Following previous works (Somepalli et al. 2023a b; Chen et al.) 2024), we use the Self- Supervised Descriptor (SSCD) score (Pizzi et al., 2022). designed to detect and quantify copying in DMs, for sim- ilarity computation in this paper. Intuitively, AS measures how well the extracted dataset X covers the images within the training dataset Xo-
Recent studies (Wen et al. 20246 Ren et al., 2024) fur- ther explored to mitigate the memorization issue. How- ever, their inference-time parts do not weaken our method where full model weights are accessible. Additionally, their training-time parts rely heavily data selection, mak- ing them impractical for personalized fine-tuning scenarios with limited data availability.
Metric 2: Average Extraction Success Rate (A-ESR). Following Carlini et al.] (2023), when the similarity be- tween an extracted image and a training image exceeds a given threshold, the extraction of that image is considered as successful. To assess the extraction of an entire dataset,
3

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
compute the average extraction success rate follows:
Recalling the equivalence between denoisers and the score function in DMs (Vincent, 2011). employ time- varying noising process and represent each score de- noising prediction, denoted by e(x,, t), similar to previous work (Gandikotta et al. (023):
A-ESR,(Xo, 8)
Σ 1
Xoo
(
(max
sim( sim( sim(Xo) (0) >
Xo
(2) where 1 is the indicator function. Following previous work (Somepalli et al. 2023a|b b). the threshold is set to 0.7 for strictly successful extraction. We also present re- sults where the threshold 7 is set to 0.6, which represents a moderate similarity and can be considered a loosely suc- cessful extraction (Chen et al. 2024).
<g (1t 1) 60. (at, 1) + (w
(co, (6)
where w 1/A. Eq. 6 demonstrates that by extrapolat- ing from the pretrained denoising prediction ( f)) to the fine-tuned denoising prediction cør (Iz, t), we can de- rive guidance toward the fine-tuned data distribution. We call this process "model guidance". The guidance scale w should be inversely related to the number of training iter- ations. With model guidance, we can effectively simulate "pseudo-" " denoiser Cq. which be used to steer the sampling process toward the high-probability region within
4. Fine Extracting Fine-tuning Data
In this section, we introduce FineXtract, framework de- signed for robust extraction using DMs before and after fine-tuning- As shown in Fig. 2 we first address sim- plified scenario considering unconditional DMs (Sec. [4.1. Next, we explore the case where the training caption e is provided (Sec. 4.2). Finally, we apply a clustering algo- rithm identify the images with the highest probability of matching those in the training dataset (Sec. 4.3) from gen- erated image set X. The output of the clustering algorithm serves as the extracted image set X, closely resembling the training images set Xo.
fine-tuned data distribution g(x).
4.2. Guidance with Training Caption Provided
We further consider the scenario where DMs are fine-tuned with a given caption c. As discussed in previous work on classifier-free guidance (CFG) (Ho & Salimans 2022), DMs often struggle to accurately learn the conditional dis- tribution and therefore require additional guidance from unconditional generation. We can adopt a similar approxi- mation to the one presented in Sec. 4.1
4.1. Model Guidance
We denote the the fine-tuned data distribution as q(x) for a fine-tuning dataset Xo. During the fine-tuning process, the DMs progressively shift their learned distribution from the pretrained DMs' distribution (2( toward the fine-tuned data distribution q(x).. Thus, we parametrically approxi- mate that the learned distribution of the fine-tuned DMs, denoted as por (z), satisfies:
Po(xlc) x (2 (zlc),
(7)
where (x(x)c) denotes the data distribution conditioned on c. The above formulation indicates that conditional DMs learn a mixture of the conditional distribution of real data and the unconditional distribution of DMs. To capture the score of a denoiser Ego (xx c). which guides sampling toward the high-probability region of go(a))c), we follow the tran- sition from Eq. 5 to Eq. 6 using denoising prediction to represent the scores:
Po (x) Po 1- (z)
(3)
where À is a coefficient ranging from 0 to I, relating to the training iterations. More training iterations result in larger X, showing the fine-tuned DMs distribution pe (z) more closely ensemble fine-tuned data distribution (1x).
Ego (It, t, c) co (wW co) t)),
(8) where w' = 1/X. This results in CFG with guidance scale w' (Ho & Salimans 2022). Furthermore, for fine-tuned DMs 8', we similarly obtain:
In this case, we can derive the score of the fine-tuned model distribution po (I) by:
V log (x) = (1-A)V Log (2)) log q(x). (4)
po Po ()) x pp-N(())
(x))² (xlc),
This means that we can derive the guidance towards the fine-tuning dataset Xo by using the score of the fine-tuned data distribution and pretrained DMs distribution:
where q(</ec denotes the fune-tuned data distribution con-
ditioned on e. Combined with Eq.
Per (x/c) x P
(1-A)(1-X)) (x)g (1-x) (x/c). (10)
Vz log log (x) 1
Vx log (()) = =
Va log (x).
X
(5)
This implies that:
4

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Extracted Images X
Closely Resemble!
Training Images X
Clustering
Sec. 4.3)
Pretrained DM
(x N)
few-shot fine-tuning
Guided Generation
Generated Image X,
Generated Image Set X
(Sec. 4.1, 4.2)
Fine-tuned DM 0
Training Caption
Hidden from Attacker
Accessible to Attacker
Figure 2: Our framework FineXtract, extracting training images using DMs before and after fine-tuning.
the first few trainable linear projection layers before and after fine-tuning. Details are provided Appendix Sec.
cor (2,, =(1 A)(1 - 1)(1-)X111)))) ()) A(1 (2,,22,, + (X,,)(
(11)
4.3. Clustering Generated Images
Sections 4.1 and 4.2 explain how to sample images within high probability region of fine-tuned data distribution. However, the randomness in the sampling process affects the images, reducing extraction accuracy. To further im- prove extraction accuracy, take inspiration from previ- ous work (Carlini et al. 2023), sampling N images and applying clustering algorithm to identify the images with the highest probability, where N > ber of training images.
Since the real-data distribution involving two modalities is expected to be more peaked than single-modality distri- bution, assume that the conditional fine-tuned data dis- tribution q(x)jc) e) is also much more concentrated than the unconditional one, y(z:). This results in significant differ- ence in the magnitude of their score, i.c., | log ((r)] << V|V log (xx I| Consequently, based on the transforma- tion in Eq. 5 and Eq. [ we have (x, 1) < Eg(2, I, c). allowing us to approximate Eq. by omitting by omitting ea (xx
Ng and N, is the num-
Specifically, inspired by Carlini et al. 2023). compute the similarity between each pair of generated images and construct graph where each image is represented as vertex. We connect two vertices when the similarity be- tween the corresponding images exceeds a threshold é, i.e., if (1; x;) 2 o, we connect vertices and j. By default, we use SSCD (Pizzi et al. 2002) to measure similarity, in line with previous work (Somepalli al. 2023a b). Instead of using fixed threshold (Carlini et al. 2023). gradu- ally increase the threshold until the number of cliques, each denoted by (A), within the graph identical to the pro- posed number of training images No. This approach helps us identify the generated image subset (i.e., the clique ((k) corresponding to each training image (xx(a)). Next, identify the central image j((k) for each clique (Ak) The central image is defined as the one that maximizes the av- erage similarity with the other images in the clique: ((A) = arg max T Ex, sim(ar, J.y). The final extracted in image set is represented as X (2:00) (1)), ((No))
egr ( - X()e((zrt)) t) + X (xr, t, c),
(12)
which indicates:
c)( xeer (2,, c) + (w' (x,,, c0 t))
+ +kgl(at
(13)
Here, = = and l: _0'1
This transformation demonstrates that can guide gen- eration within the conditional fine-tuned data distribution, ((z]c), by extrapolating from the unconditional denoising prediction of the pretrained DM, co(2,, to the condi- tional denoising prediction of the fine-tuned model DM, egr (2 t, c), using the guidance scale w'. This process also involves additional correction term keo (zt, t), which, in- tuitively, compensates the mismatch between model guid- ance and CFG.
In practice, the training caption may not always be avail- able. However, we find that it is possible to extract some information about the training caption by analyzing only
Intuitively, our clustering algorithm seeks to find the subset of extracted images corresponding to each training image
5

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Table 1: Comparison of FineXtract and other baseline methods in fine-tuning data extraction for style-driven generation using WikiArt dataset [Nichol] 2016) and for object-driven generation under Dreambooth dataset (Ruiz et al. (023) under different fine-tuning methods. A-ESR and p- refer to the Average Extraction Success Rate, with the threshold for successful extraction set 0.6 and 0.7, respectively. The experimental results demonstrate that FineXtract exhibits stronger performance under all scenarios and metrics than baselines.
Style-Driven Generation: WikiArt Dataset
DrcamBooth
LoRA
Metrics and Settings
ASt
R-
A-ESRa.s1
ASt
A-ESR
A-ESRo.61
Direct Text2ing+
0.317
0.00
0.01
0.299
0.00
0.00
CFG+Clustering
0.396
0,03
0.11
0.357
0.00
0.01
FineXtract
0.449
0.06
0.22
0.376
0.01
0.05
Object-Driven Generation: DreamBooth Dataset
DreamBooth
LoRA
Metrics and Scttings
AS
A-ESRp.7
A-ESRp.st
ASt
A-ESR
A-ESRø.61
Direct DextZing-Clustrriin
0.418
0.03
0.11
0.347
0.00
0.02
CFG+Clustering
0.528
0.15
0.36
0.379
0.01
0.05
FineXtract
0.557
0.25
0.45
0.466
0.04
0.18
and then identifies the central image within each subset.
method's ability extract the exact training dataset. For DreamBooth, the guidance scale w' for both FineXtract and CFG set to 3.0 by default, with the correction term scale k set to -0.02 in Equations 8 and 13 For LoRA, w' is set to 5.0 for FineXtract and 3.0 for CFG, respectively. For the clustering algorithm, we by default set the maximum clus- tering time for each threshold be 30s. If clustering does not end, we simply move to the next threshold reduce computation time. We discuss how these hyperparameters influence extraction efficiency in Sec. 5.3 FineXtract un- der potential defenses and toward real-world checkpoints on HuggingFace are discussed in Sec. respectively.
5. Experiments
In this section, we apply our proposed method, Fine tract, to extract training data under various few-shot fine- tuning techniques across different types of DMs. We con- duct experiments two common scenarios for few-shot fine-tuning: style-driven and object-driven generation. For style-driven generation, which focuses on capturing the key style of set of images, we randomly select 20 artists, each with 10 images, from the WikiArt dataset (Nichol) (016)) For object-driven generation, which emphasizes the details of a given object, we experiment on 30 objects from the Dreambooth dataset (Ruiz al. 2023), each consisting of 4-6 images. This setup aligns with the recommended num- ber of training samples in the aforementioned fine-tuning methods (Ruiz et al. 2023) Hu et al. 2021). We experi- ment with two most widely-used few-shot fine-tuning tech- niques: DreamBooth (Ruiz et al. 2023), and LoRA (Hu al., 202T). More details for the fine-tuning setting are available in Appendix Sec. D
5.4 and Sec. 5.5
5.1. Comparison
Previous extraction methods primarily focus on the gener- ation capabilities text-to-image DMs, employing either direct text-to-imas -oogemage generation or classifier-free guidance (CFG) (Carlini al. 2023 Somepalli al., 2023a b). To better demonstrate the effectiveness of our framework, compare FineXtract with Direct Text-to-Image and CFG, both combined with the clustering algorithm proposed in Section 4.3 For both CFG and FineXtract, we set the guid- ance scale w' to 3.0 under DreamBooth fine-tuning. Under LoRA fine-tuning, w' are set to 3.0 for CFG and 5.0 for FineXtract. These hyperparameters are found to perform well (see Sec. 5.3 for details). All methods use the same number of generation iterations, N, set to 50 No. and the number of extracted images set to No to ensure fair comparison. The results, shown in Table demonstrate a significant advantage of FineXtract over previous methods, with an improvement of approximately 0.02 to 0.05 in AS
The default model used for training is Stable Diffusion (SD) V Additionally, we demonstrate the adaptabil- ity of our method to various types and versions of DMs, larger training datasets, and different numbers of generated images (refer to Sec. 5.2 for more details).
By default, we the generation count N to 50x 50xNo. where No represents the number of training images. The number of extracted images is set equal to No to best evaluate our
6

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Dine
CRG
FineXtmet
Trinnng Imng
DirecTextZimg
CTG
FineXtmet
Training
DreamBooth Fine-taning
Dremmäcoth Fine-tuning
DreamRooth
DreamBooth Fine-tuning
Fine-cunize
Fine-tuning
euning
Lorn
Fine-tuniae tuning
Fine-tuning
(a) Comparison WikiArt dataset.
(b) Comparison DreamBooth datasct.
Figure 3: Qualitative comparison of the extracted result between Finextract and baseline methods. All baselines are combined with the clustering algorithm proposed in Sec. 4.3
Generation GPU memory costs and time costs for per-image per-image generation using FireXtract and CFG.
Table 2:
Perec
0.50
CPG
Metria and Settnnss
DreamBouth
LeRA
2
Memory Costs (MB))
Time Costs
Memory Costs (MIS)
Time Costs
l'neXtmet CFG
18812
3.4
15594
3.6
FineXtract CPG
12096
3.4
12226
3.6
10
20
30
00
20
of training imagos
of Gimeentedd Imagoe por Toxtiing Image
and a doubling of the A-ESR in most cases.
(a) Different No
(b) Different N
Our method, as detailed in Sec. 4.1 and 4.2 employs guid- ance between two distinct models at each generation step without slowing down the generation speed compared to traditional CFG (which also requires guidance by forward- ing the main UNet in DMs twice). However, GPU memory costs increase due to the need to load both the pretrained and fine-tuned models. In Tab. 2 we present a demo ex- periment comparing the computational costs of FineXtract and CFG, using SD (v1.4) fine-tuned with DreamBooth and LoRA on the WikiArt dataset. The batch size is fixed 5, and all experiments conducted on single A100 GPU.
Figure 4: Experiment on generalization ability of FineX- tract across different number of generated images and train- ing images. We experiment on 4 classes of WikiArt fine- tuning on SD (V1.4) with DreamBooth. FincXtract consis- tently outperforms baseline under different N and No.
Models: Stable Diffusion Model (Rombach et al. 2022). Stable Diffusion Model XL (Podell et al.) 2023), and AltD- iffusion YYe 2023), which are representative of latent- space DMs, high-resolution DMs and multilingual DMs, respectively. We conduct experiments using the following versions of the three models: SD (V1.4)3 SDXL (V1.0 and AltDiffusiorf] As shown in Tab. 3 the improvement of our method compared to the baseline is consistent across different DMs.
5.2. Generalization
In this section, we take a step further test whether our method can be applied to broader range of scenarios, in- cluding different DM structures, varying numbers of train- ing images No. and different numbers of generated images N. We experiment on classes in WikiArt dataset fine- tuning DMs with DrohhBooth.
1.0
Different DMs. We select three distinguishable Diffusion
httg://haagiingfcccccc/AAAAAAAAt

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
3'=3.0
4/0
10.0
Triining Image
Figure 5: Visualization of generated images using FineXtract under different w/ with fixed xr.
Table 3: Experiments of Fine different DMs. We experiment on 4 classes in WikiArt dataset using Dream- Booth for fine-tuning. The guidance scale w' for both CFG and FineXtract is set to the value that achieves the highest AS for each DM.
0.5
0.45
0.45-
3
0.35
PineXtract
8.42
FiaeXtraci
Guithaus
-0.00
6:22
0.M4
Metrics and Settings
AS1
(V14)))
A-E5Raa7
AS1
A-ESRaaT
(V1.0)
AS?
AltDifuusion
A-ESR;47
(a) Ablation on w
(b) Ablation on k
Direct ClustengCu
CHGrClusering
0,34]
0.434
0.23
0.03
0.335
0.360
0.05
0.252
0.364
0.00
FineXtract
0,501
0.35
0.467
0
Figure 6: Ablation Study on hype-parameters w' and k. We experiment 4 classes of WikiArt dataset fine-tuning SD (V1.4) with DreamBooth.
Table 4: FineXtract under defenses.
Defense Methods
AS1
A-ESRg.et
Number of Training Images No- As the number of training images increases, the learned concept during fine- tuning becomes more intricate, thereby enhancing the dif- ficulty of extracting training images. To thoroughly exam- ine how this influences performance, we conduct experi- ments with varying numbers of training images in 5 classes in WikiArt, the results of which are presented in Fig, 4a where we can observe performance drops when the num- ber of the training images is large.
No Defense 0.501
0.35
Cutout 0.397
0.08
Rand.Augment
0.267
0.03
is too low, the guidance provided by fine-tuning methods is weakened. Conversely, if w' is too high, it often causes generation failures, resulting in unrealistic outputs (see vi- sual examples in Appendix Fig. [5) As shown in Fig. 66a w'' = 3.0 works well for both CFG and Fire when DMs are fine-tuned using DreamBooth.
Different Numbers of Generated Images N. As previ- ously mentioned, the clustering algorithm allows attackers to leverage more generation iterations to improve extrac- tion accuracy. We test method with different numbers of generated images N, ranging from No to 50 x No. As shown in Fig. 4bb] increasing N significantly improves per- formance. However, the time complexity of finding max- imal cliques can grow exponentially with the number of nodes (Tomita et al., 2006). Thus, further increasing N to larger than 50x No makes considerably more difficult for the clustering algorithm to converge.
Eq. [3] we derive a correc- tion term (e, t) to address the inconsistency between CFG and model guidance. Although Eq. [3] indicates that k should not be less than O, our experiments suggest that setting k = 0000 typically yields the best results, as illus- trated in Fig. 6bb This may be due to the complex interac- tion between w and k. where can only be guaranteed to be greater than if the true value of w² can be identified, which often impractical in real-world scenarios.
Correction Term Scale
5.4. FineXtract under Defense
5.3. Ablation Study
As highlighted in prior research (Duan et al. 2023 Kong et al. 2023 Pang et al. (023). is possible to partially defend against privacy-relate attacks, such membership inference attacks (MIA).
In this section, we experiment with hyperparameters in Eq. 13 including the guidance scale w' and the correction term scale As We experiment on 4 classes in WikiArt fine-tuning with Dooth Results under LoRA are shown in Ap- pendix Sec. C
Naturally, this raises the question of whether these defense methods can also protect against extraction techniques. To explore this, we conducted experiments on FineXtracc un- der two defenses: Cutout (DeVries & Taylor 2017), and
Guidance Scale w' The guidance scale w' is the most crit- ical hyperparameter influencing extraction efficiency. If w/
8

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
No Defense
Cutout
RandA Augment
consistently outperforms the baseline methods, increasing AS by at least 0.03 and doubling A-ESR in most cases.
6. Conclusion
In conclusion, our proposed framework, FineXtract, effec- tively addresses the challenge of extracting fine-tuning data from publicly available DM fine-tuned checkpoints. By leveraging the transition from pretrained DM distributions to fine-tuning data distributions, FineXtract accurately guides the generation process toward high-probability re- gions of the fine-tuned data distribution, enabling suc- cessful data extraction. Our experiments demonstrate the method's ' robustness across various datasets and real- checkpoints, highlighting the potential risks of data leakage and providing strong evidence for copyright infringements.
Figure 7: Images generated by SD (V1.4) fine-tuned under various scenarios, where noticeable decline in quality can be observed when defenses are applied.
A Acknowledgments
Table 5: Comparison of FiraXtract with baseline methods towards real-world checkpoints.
We sincerely thank Peishen Yan, Yuxiang Lu, Xirui Li, and Suizhi Huang for their valuable feedback and suggestions on this paper.
Metrics Settings
A-ESR.
DneamBlooth
A.ESR..
AS1
A-ESR,
LoRA
Direct TexTing Cluseriin
0.562
8.00
0.00
0.270
CFG
0.458
0.04
0.20
Impact Statement
FincXract
0,333
0L33
0.38
0,371
This paper explores methods for extracting fine-tuned data from publicly available checkpoints. The proposed ap- proach offers a dual impact: it can serve as a legal tool to detect unauthorized data usage, while also raising aware- ness about the risks of data leakage for those who share fine-tuned models online. These insights are intended to contribute to advancing responsible practices in model sharing and checkpoint distribution.
RandAugment (Cuuuk al.] 2020p. Notably, RandAug- ment is recognized strong privacy-preserving defense at the cost of severe decline in generation quality (Duan lt al. 2023
The results presented in Tab. 4 illustrate how these meth- ods can partially defend FineXtract, though at the cost of generation performance. Cutout and RandAugment indeed proves to be quite strong at defense. However, as shown in Fig. 7 the added transformations render the output im- ages largely unusable, making them difficult to leverage in practice. Quantitative measurements of this unusability are provided in Appendix Section | Our results highlight that while these approaches may be partially effective in de- fense, there is lack of research on how to fine-tune models on such transformed data while preserving the defensive ef- fects. This remains an area for further investigation.
From an ethical perspective, this work highlights the need for caution and transparency in model tie-ttnnng and shar- ing, particularly concerning sensitive or proprietary data. The societal consequences of this work include promoting accountability in data usage while underscoring the impor- tance of copyright protection and privacy in machine learn- ing. However, we acknowledge that potential attackers may misuse the techniques described in this paper, which could pose significant threats. By revealing these vulnerabilities abead of time, this paper aims to inspire the development of future defense methods safeguard against such risks.
5.5. Real-world Results
Finally, we test our method on fine-tuned checkpoints available in the real world. We experiment on 10 check- points from HuggingFace where the corresponding train- ing datasets are provided, allowing to evaluate the ef- fectiveness of our extraction method. Due to licensing re- strictions, we only provide detailed information about the checkpoints with permissive licenses in Appendix Sec. E Quantitative results are shown Tab. 3 where FineXtract
9

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
References
Jiang, H. H., Brown, L.., Cheng, J., Khan, M., Gupta, A., Workman, D., Hanna, A., Flowers, J., and Gebru, T. Ai art and its impact on artists. In Proceedings of the 2023 AAAI/ACM Conference on Al, Ethics, and Society, PP. 363-374, 2023.
Civitai: The Home of Open-Source Generative AL
http::/wwwwccviita.coom
Hugging Face: The Al community building the future. https//huggingffacecoo
Kawar B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., and Irani, M. Imagic: Text-Based Real Image Editing With Diffusion Models. arXiv preprint arXiv:2210.09277 2022.
Abdi, H. and Williams, L. J. Principal component analysis. Wiley interdisciplinary reviews: computational statis- tics, 2(4))433-455 2010.
Kong, F., Duan, J., Ma, R., Shen, H., Zhu, X., Shi, X., and Xu, An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization. arXiv preprint arXiv:2305.18355 2023.
Bertran, M., Tang, S., Kearns, M., Morgenstern, J., Roth, A., and Wu, Z. S. Reconstruction attacks on machine un- learning: Simple models are vulnerable. arXiv preprint arXiv:2405.20272. 2024.
Liang, C., Wu, X., Hua, Y., Zhang, J., Xue, Y., Song, T., Zhengui, X., Ma, R., and Guan, H. Adversarial Example Does Good: Preventing Painting Imitation from Diffu- sion Models via Adversarial Examples. In ICML, 2023.
Carlini, N., Hayes, J.. Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., and Wallace, E. Ex- tracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253-5270, 2023.
Nichol, K.
Painter by Numbers, WikiArt. 2016.
Chen, Y., Ma, X. Zou, D., and Jiang. Y.G.. Extracting training data from unconditional diffusion models. arXiv preprint arXiv:2406. 12752, 2024.
Pang, Y., Wang, T., Kang, X., Huai, M., and Zhang, Y.
White-box Membership Inference Attacks against Dif- fusion Models. arXiv preprint arXiv:2308.00 06400, 2023.
Cubuk, E. D., Zoph, B., Shlens, J., and Le, Q. V. Randaug- ment: Practical Automated Data Augmentation with Reduced Search Space. In CVPR, 2020.
Pizzi, E., Roy, S. D., Ravindra, S. N., Goyal, P., and Douze, M. A self-supervised descriptor for image copy detec- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14532- 14542, 2022.
DeVries, and Taylor, G. W. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv preprint arXiv:1708.04555 2017.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dock- horn, T., Müller, J., Penna, J., and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Duan, J., Kong, F., Wang, S., Shi, X., and Xu, K. Are Diffusion Models Vulnerable to Membership Inference Attacks? arXiv preprint wwx.i:2302..13110 2023.
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An Im- age Is Worth One Word: Personalizing Image Generation Using Textual Inversion. arXiv preprint arXiv:2208.016186 2022.
Qiu, Z., Liu, W., Feng, H., Xue, Y., Feng, Y., Liu, Z., Zhang, D., Weller, A., and Schölkopf, B. Controlling Text-to-Image Diffusion by Orthogonal Finetuning. In NeurlPS, 2023.
Gandikota, R., Materzynska, J., Kauftan J., and Bau, D. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1426-2243, 2023.
Ren, J., Li, Y., Zeng, S., Xu, H., Lyu, L.., Xing, Y., and Tang. J. Unveiling and mitigating memorization text- to-image diffusion models through cross attention. In European Conference Computer Vision, pp. 340-356. Springer, 2024.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12552 2022.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High- nigh-Resolution Image Synthesis With La- tent Diffusion Models. In CVPR, 2022.
Ho. J., Jain, A., and Abbeel, P. Denoising Diffusion Prob- abilistic Models. In NeurlPS, 2020.
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. DreamBooth: Fine Tuning Text-to- Image Diffussin Models for Subject-Driee Driven Generation. CVPR, 2023.
Hu, E. J., Wallis, P., Alle Z., Li, Y., Wang, S., Wang. L., Chen, W., et al. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR, 2021.
10

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Shan, S., Cryan, J.. Wenger, E., Zheng, H., Hanocka, R., and Zhao, B. Y. Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222,, 2023.
Yang, R., Srivastava, P., and Mandt, S. Diffusion Proba- bilistic Modeling for Video Generation, arXiv preprint arXiv:2203.0944 2022.
Ye, F., Liu, G., Wu, X., and Wu, AltDiffusion: A Mul- tilingual Texa-to-Image Diffusion Model. arXiv preprint arX.v:2308.0990 2023.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, Deep Unsupervised Learning Using Nonequilibrium Thermodynamics. In JCML, 2015.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Diffusion art digital forgery? investigat- ing data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6048-6058, 2023a.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Understanding and mitigating copying in diffusion models. Advances in Neural Information Pro- cessing Systems, 36:47783-47803. 2023b.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In NeurlPS, 2019.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, Score-based Generative Mod- eling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13455 2020.
Tomita, E., Tanaka, A., and Takahashi, H. The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical computer sci- ence, 31: 42, 2006.
Vincent, P. A connection between score matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Wang, J., Chan, K. C., and Loy, C. C. Exploring Clip for Assessing the Look and Feel of Images. AAAI, 2023.
Wen, Y., Jain, N., Kirchenbauer, J., Goldblum, M., Geip- ing, J., and Goldstein, T. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Process- ing Systems, 36, 2024a.
Wen, Y., Liu, Y., Chen, C., and Lyu, L. Detecting, explain- ing, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learning Rep- resentations, 2024b.
Wu, X., Hua, Y., Liang, C., Zhang, J., Wang, H., Song, T., and Guan, H. Cgi-dm: Digital copyright authentication for diffusion models via contrasting gradient inversion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 10812-10821, 2024.

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
A. Caption Extraction Algorithm
While it can be argued that training captions may not always be available, we find that they can be partially extracted. Our focus is on the first layer that is not frozen during the fine-tuning process. We assume this layer behaves as linear model without bias, which aligns with the common scenario when fine-tuning DMs. Specifically, in the case of fine-tuning SD using LoRA, the LoRA is typically applied to the cross-attention layers. As result, the first layer that is fine-tuned is the linear projection layer in the cross-attention module, which processes the text features from CLIP model based on the input prompt. This assumption also holds true when fine-tuning using DreamBooth without adjusting the text encoder, which is one of the most frequently used fine-tuning settings.
The weights of this layer before and after fine-tuning are denoted as 3r and B The output of the layer for given input prompt embedding is Age". Unlike prior work (Bertran et al. 2024), we do not have clear formulation of the training target for this particular linear layer, as the downstream signal can change frequently. Therefore, in this case, we rely on the gradient updating process.
A.1. A Basic Scenario
To begin with, let's consider a very simple case where the prompt consists of only one word, and all the training images share the same training caption. We denote the embedding of this specific prompt as cu- eu has the shape [1, M] for SD, where N is the dimension for the embedding (we omit the positional embedding here and will discuss it later). The weight 3x for the projection layer is with shape [H, NN, where H is the hidden dimension. Then the forward loss is L e The gradient can be computed with:
aL(S e)
0L(55 e
Bx
03%
033 o
ED.
(14)
During the jth update, we denote this as V 87 ol (3 ). Then for basic optimizer, such as SGD, we have:
B - B% =
(3 eoo)
(15)
which means the row space for the matrix St Br is in fact span(co): With this information, we can simply use a different embedding of to index this equation:
- = Σ o
coo!
(16)
scalar
Notably, if all e; are normalized, we should have arg max e = Eg. Therefore, we can find this e; by simply finding the one that maximizes the norm in Eq. 16
Co arg max
||0 - - -e 11.
(17)
e
A.2. Extension to Multiple-Words Prompts
In general cases, prompts consist of multiple words, making the inversion process tricky. In these cases, eo may have the shape [W, N] for SD, where W is the length of the prompts. (In fact, due to the presence of position embedding, cases with different words actually all have W 77, where 77 is the maximum length for input prompts). This results in eoe; not being scalar anymore in Eq. 16 Its shape is []. Therefore, we cannot obtain eo by simply computing the norm. In fact, Eq. 15 shows that the row space of / - 9h is span (eo.1, e0,2, 60.77. Here, we can use a transformation where we perform Principal Component Analysis (PCA) decomposition (Abdi & Williams 2010) on 3t - - In other words, we approximate using a rank-one matrix: PCA 8 )= = Lg where to has the shape [1, Nj. This can also be regarded as finding the main vector direction within the space span C0,1, 60,2, ". Co,77) With this decomposition, we then have:
12

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
eo arg max (PCA -5
(18)
A.3. Extension More Optimizers and Approximation During Training with Multiple Linear Matrices
In more general cases, Adam is usually used instead of SGD, resulting in Eq. not holding exactly but with some error. Moreover, we use some adaptors for fine-tuning, such equations also have some error as we are trying use low-rank matrix to approximate higher-rank matrix. Therefore, the row space for 1 3g may now be (ean + 6,,1 60, + ; (k,77), where is small error term.
We can incorporate more information to reduce the effect brought by the error term. In practical scenar- ios, we have many linear projection layers that accept the same text embedding as input. For example, in SD, we may have about 20 such layers. Therefore, we can concatenate all of them together and try to find a decomposition considering them all.
other words, find a main vector for all matrices within the space span c0 C0,2; ... C0,77, 60,1 60.2.-.- 60,77: C1,1, ,61,77,"' ,K,1, ,K,77], which corresponds PCA[[( B (B - B (3x Bx)]).
However, in practice, we find that such PCA may fail in most cases, suffering from performance drop. This may be due to the fact that the error term is not necessarily insignificant. Therefore, we perform PCA with S and & respectively. Then we compute the difference between them, i.e., PCA(3+) PCA(3-) instead of performing PCA directly on, i.e., PCA(B+ B-))
eg arg max (ex) = arg max (PCA| Bt, .B.l PCA.BT.81 .T.... ,Kklee 6t
(19)
The intuition behind applying PCA to the row space is that extracts the most significant signals from the training prompts. For a pretrained model 3 such signals represents the principal vector for real-world prompts. For fine-tuned model S+,, such signal should also highly align with real prompts, but with some information about fine-tuned captions. Using PCA first in Eq. 19 can prevent extracted embedding from diverging into unrelated or complex signals and staying within the embedding space of real-world prompts.
A.4. Optimization
Optimization over Eq. 19 can indeed derive some prompt embeddings, but these may not be related any real prompts. So the real optimization target should be finding a input prompt co:
R arg max F(E(c))).
(20)
where E(-) is the frozen text feature embedding function. The challenge that the prompt space is discrete, which turns the optimization problem into hard-prompt finding task. To overcome this, we adopt the technique from Wen al.] (2024)): during optimization, we project the current embedding onto a real prompt and use the gradient of this real prompt's embedding to update our given embedding. Algorithm iillsstrates this framework.
Algorithm Hard Prompt Extraction on Fine-tuned Caption for Linear Layers
Input: Pre-trained Linear parameters 3 fine-tuned Linear parameters 8t, input caption co, text encoder E, optimiza- tion iterations N, learning rate a, projection function E Output: Extracted caption é.
Initialize embedding é with random prompts: F(()..
for i 0 to N do
é == --=--11 (2)
Find gradient 8 = V Ele) F(E(2)) based on this é, - and 3 using Eq. 19 Update é + é + ad
end for
3- E é-E (2)
13

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
A.5. Experiment Result
We conduct experiments on four classes of the WikiArt dataset, where the DMs are fine-tuned using DreamBooth. Table presents comparison between the original training captions and the captions extracted by our method. The results demon- strate that representative information can be extracted to some extent, indicating leakage of training captions information based on the DMs before and after fine-tuning.
Table 6: Experimental results of our caption extraction algorithm using the L2-PGD attack with 1000 iterations starting from a random prompt. This extraction costs approximately 1.5 to 2 minutes per sample on single A100 GPU. The results demonstrate that key information, such as the artist's 's name, can largely be inferred. Correctly inferred parts are shown in bold.
Training Caption
Extracted Caption (s words)
Extracted Caption (s7 words)
art style Post Impressionism vincent art style Fauvism henri matisse art style High Renaissance leonardo da vinci art style Impressionism claude monet
attributed impressionism vincent
plein impressionism vincent demonstrating! fantastic !)) donncinarte hemingway henri matisse throughout fincart paintings lconardo onda elengrembrandi pre picasso artwork
henri paintings matisse
Iconardo confident paintings
suggestions impressionism monet
cassini gustave monet impressionist monet etosuggesti
Experiment results under longer training prompt are discussed in Sec. J
We further evaluate our method using extracted captions (3 words in length) as shown in Tab. 7 While the success rate decreases compared to using full captions, it remains substantially higher than extraction without any caption informa- tion, confirming the effectiveness of our caption extraction approach. Moreover, FineXtract consistently outperforms the baseline under the extracted caption setting.
B. Results with No Prompts Provided
We also explore how unconditional generation can be used to extract training images. We conduct experiments on four classes of the WikiArt dataset, using various prompts and generating images with an empty prompt. The results, shown in Fig. 8 demonstrate that FineXtract improves performance compared to the baseline (since no conditional information is available, CFG cannot be applied, so the baseline corresponds the w' 1 case in FineXtract). However, the AS is significantly lower than when caption is provided, leading far fewer successful extractions.
0.18
0.16
FineXtract
0.15
C. Abaltion Study of w/ under LoRA Fine-tuning
Guidance Scale w/
In LoRA, we find that the suitable w' for CFG and FineXtract differs, which are 3.0 and 5.0, respectively, as shown in Fig. g We experiment on classes of WikiArt dataset fine-tuning on SD (V1.4) with LoRA.
Figure 8: Experiment result when no training prompt provided.
D. Finetuning Details
The details of the parameters in the fine-tuning methods are presented below. We use No to represent the number of images utilized for training. Our setting mostly follow the original training setting in few-shot fine-tuning methods (Ruiz et al. 2023).
0.40
0.38
0.36
0.34
FineXtmatt
Dreambooth: We use the training script provided by Diffuser: Only the U-Net is fine-tuned during the training process. By default, the number of training steps is set to 200 Ny, with a learning rate of 2 x 10-6 The batch size is set to 1. We set the prior loss weight as 0.0 for simplification. For the WikiArt dataset. the instance prompt is "art style of [class name]", where (class name] is the name of the artist such as "Fauvism henri matisse".. For the Dreambooth dataset, the
CPG
0.32
Guidance Scale w
Figure 9: Ablation Study on hype- parameters w'.
wwwpy//wwhnnnooooooooooooooooeee dreambooth.py
14

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Table Extraction comparison on fine-tuned SD vl.4 checkpoints using DreamBooth under different caption settings.
Extraction Method
AS
A A-ESRo.6
FineXtract (Full Caption)
0.501
0.35
FineXtract (Extracted Caption)
0.314
0.15
FineXtract (Empty Caption)
0.192
0.00
CFG + Clustering (Full Caption)
0.434
0.23
CFG + Clustering (Extracted Caption)
0.308
0.08
Direct Text2lmg + Clustering (Empty Caption)
0.146
0.00
Table 8: Information about subsets of the checkpoints used for experiments.
Model Name
SD Version
Fine-tuning Methods
# of Training Image
stlddrcamboooh-ibbrryymaaaaa
SD (V1.5)
DreamBooth
5
sd-dre
SD (V1.5)
DreamBooth
5
SD (V1.5)
DreamBooth
Norof785SDXL-YaamAtrryy---LLL
SDXL (V1.0)
Lora
Norod78/ppokir-sshk
SDXL (V1.0)
Lora
22
Nano777/DDXL-PrrngeesTbbbooo
SDXL (V1.0)
Lora
138
instance prompt is "a [class name]" where [class name] is the class of the object, such "dog".
All default parameters remain consistent with the case in Dreambooth (No Prior), with the exception of the learning rate, which adjusted to 1 x 10 The rank is fixed to to ensure the fine-tuning process capture fine- mne-grained details of training samples. The prompts used are the same as the case DreamBooth.
LoRA: We use the training script provided by Diffuser +
E. Real-World Experiments Setup
We randomly selected 10 fine-tuned DM checkpoints from Hugging Face, which include those fine-tuned from SD (V1.5) and SDXL (V1.0) using DreamBooth and LoRA. The number of training images ranges from 5 to 138. We compute the and A-ESR for each checkpoint and average the results, which correspond to those presented in Tab. [S] Details of these checkpoints, all with permissive licenses, are provided in Tab. These checkpoints are available on Hugging Face for result reproduction. We sincerely appreciate the checkpoint creators for sharing their work support both research and practical applications.
F. Experiment on Mixture of Dataset
F.1. Mixture of DreamBooth and WikiArt Datasets
We constructed new dataset with 10 classes, each containing images from DreamBooth and 5 from WikiArt. As reported Tab. 9 this mixture led to mutual decreases the fine-tuned model's fidelity (measured by DINO score RRuzz et al., 2023),, image quality (measured by CLIP-IQA (Wang et al., (023)), and extraction rate. Therefore, extracting training images from this model becomes more challenging. Nonetheless, FineXtract still significantly outperforms the baseline despite the fact that its performance partially drops compared the original cases, further verifying its generality and effectiveness.
wwwprp/miii.nmonngooooooooooooom dreambooth lora.py
15

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Table 9: Comparison of the performance of FineXtract and the baseline in line-tuning data extraction under separated and mixed data scenarios using DreamBooth and WikiArt datasets. We observe mixed data makes extraction more challenging while also making harder for diffusion models to learn the given data, evidenced by lower fidelity (DINO) and reduced image quality (CLIP-IQA).
Dataset
Extraction Method
ASt
A-ESRu.6f
DINO
Clip-IQA t
CFG+Clustering 0.525
0.45
Separated Data
0.533
0.697
FineXtract 0.572
0.55
CFG++Clstering 0.457
0.08
Mixed Data
0.266
0.447
FineXtract 0.480
0.18
Table 10: Comparison of the performance FineXtract and baseline in fine-tuning data extraction under dataset mixed from different number of classes of WikiArt dataset. Mixed data with different classes makes model learning poorer, causing lower fidelity (DINO) and image quality (Clip-IQA), which in turn makes extraction more challenging.
Mixed number of Artists Per Class
Extraction Method
ASt
A-ESRo.st
DINO 1
Clip-IQA f
CFG+Clustering
0.396
0.11
1 Artist Per Class
0.458
0.525
FineXtract
0.449
0.22
CFG+Clustering
0.390
0.20
2 Artist Per Class
0.387
0.441
FineXtract 0.436
0.20
CFG+Clustering
0.353
0.15
5 Artist Per Class
0.343
0.478
FineXtract 0.388
0.23
F.2 Mixture of Different Styles in WikiArt Datasets
We conducted experiments with increasing number of classes and found that while it reduces our method's performance, we still significantly outperform the baseline. As shown in Tab. 10, we observe that the number of styles increases, the extraction success rate decreases. Additionally, we evaluated the model's ability learn the input distribution and found that higher number of classes leads to lower fidelity (DINO) and image quality (Clip-IQA), reflecting the model's difficulty in learning the fine-tuning data distribution accurately.
G. Ablation Study on Model Guidance
We provide further discussion by comparing model guidance using conditional DM, CFG, and a combination of CFG and model guidance (with k 0). As shown in Fig. 10 model guidance using conditional DM achieves close performance to the combination method and significantly outperforms CFG alone, suggesting model guidance's dominance and a potential misalignment with CFG, affecting the parameter k.
0.50
0,45
AS
0.40
FineXtract
0.35
CFG
H. Experiment Results on FLUX.
Model Guidance Only
0.30
We update the results using FLUX.1 [dev] The DreamBooth fine-tuning on FLUXX.1 [dev] requires more than 100GB per GPU, which we are currently unable to run. However, we have conducted experiments in the LoRA scenario using the official scripts provided by Diffusers. The training iterations are fixed 150, as suggested by the repository, and other hyperparam- eters remain the same those in the repo. As shown in Tab,
2
4
8
10
Guidance Scale w'
Figure 10: Ablation study on different w/ dealing with model guidance, CFG, and Fine We experiment on 4 classes of WikiArt dataset.
16

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Table 11: Extraction comparison between the baseline and our method on fine-tuned FLUX checkpoints using LoRA.
Extraction Method
ASt
A-ESRo.7 f
A-ESRo.6 1
CFG+Clustering 0.568
0.458
0.525
FineXtract
0.507
0.460
0.487
Table Image fidelity and quality for checkpoints under different defenses. DINO measures image fidelity with respect the training dataset, and thus it is not applicable when dealing with source images. We use "N.A." denote this scenario.
Source Images
Generated Images
Defenses
DINO1
Clip-IQAt
DINO1
Clip-IQAt
No Defense
N.A.
0.568
0.458
0,525
CutOut
N.A.
0.507
0.460
0.487
Rand/ugment
N.A.
0.522
0.435
0.479
11 compare method with the baseline (CFG), and both methods show the best performance when the guidance strength is set to 3.0, with the improvement being consistent.
I. More Assessments for Pre- Pre-processing Defense
We attempted to quantify usability using a no-reference image quality score and the image fidelity measurements. We follow previous works (Ruiz et al. 2023), using DINO for image fidelity measurements. For image quality measurements, Clip-IQA [Wang et al. 2023). As shown in Tab. T2 we found that the CutOut and RandAugment degraded Clip- IQA by 6.1% and 4.6% in the original dataset, and generated images experienced around 3.8% and 4.6% degradation. These extent are close in the sense that the degration brought by the transformation largely preserves in the generation process. For Rugment, the image fidelity also largely degrades. This suggests that the preprocessing steps applied to the images (removing a square section or adding high contrast to the input images) largely persist in the generated output images, hindering extraction methods from obtaining high-quality images while sacrificing generation quality of diffusson models themselves.
In summary, there exists trade-off between image quality and defensive effects for these preprocessing methods. We leave the accurate modeling of this trade-off and further improvement an interesting future work.
J. Experiment on Extracting Prompts under Different Prompt Length W
We further experiment on different scenarios involving fine-tuning with longer training prompts. Specifically, we use GPT-4 expand the original prompts into longer versions by adding information related to the author while avoiding overly specific details prevent mismatches with the input image. Tab. [3] shows extraction results. With longer text, extracting detailed information becomes more challenging. However, algorithm can still identify specific details, such as the artist's name, even with extended text. When the text length continue to increase, certain information becomes harder to extract, though such cases are rare in few-shot fine-tuning.
17

---

Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models
Experimental results of our caption extraction algorithm using the L2-PGD attack with 1000 iterations starting from random prompt. This extraction costs approximately 1.5 to 2 minutes per sample single A100 GPU. The results demonstrate that key information, such as the artist's name, can largely be inferred when the training caption is not too long. Correctly inferred key-word parts are shown in bold. Related information extracted is shown in italic.
Table 13:
Extended Prompt
Training Caption
Extracted Caption (W words)
An insight into the Post-Impresse cnst-lppresiioistt style with expressive, emotional brushwork and rich colors, inspired by the unique techniques of Vincent van Gogh.
vangogh impressionism class impressionist paintings acqu
Exploring the art style of Fauvism, characterized by vivid, bold colors and dynamic brushstrokes, as seen the works Henri Matisse.
henri matisse whose explores whose colourful stures
exploration of the High Renaissance style, marked by balance, realism, and anatomical ac- curacy, capturing the essence of Leonardo da Vinci's art.
ews inaccurate anatomy leonardo deus aron onda
A look at Impressionism, noted for soft, light- filled scenes and gentle brushstrokes, inspired by the atmospheric beauty in Claude Monet's s paint-
impressionism wgleagues monet het commence
ings.
Excessively Extended Prompt
Training Caption
Extracted Caption (< 7 words)
A deeper look into Post- Dost-Impressionism. empha- sizing Vincent van Gogh's expressive, emotional brushwork and contrasting colors. His tech- niques convey a personal, intense view of the world, moving beyond the lighter tones of Im- pressionism.
post impressionism vincent gogh pedimonet opposite van- gogh
In-depth exploration of Fauvism's vivid colors and bold, expressive brushstrokes that bring emo- tions to life, heavily inspired by the renowned techniques of Henri Matisse. His approach cap- tures the vibrant essence of Fauvism, using strik- ing colors to convey feeling.
congratulations adrian edgar forum awaits muromain destina- tion
An insightful exploration into the High Renais- sance, highlighting the balance, anatomical real- ism, and idealism of Leonardo da Vinci's cre- ations. This style embodies the Renaissance's s focus on perfection, with meticulously detailed compositions and harmonious proportions.
retains vehicles marriott alcatraspberries tuesdaythoughts shua
A detailed depiction of Impressionism, capturing transient beauty with soft brushstrokes and deli- cate light, influenced by Claude Monet's signa- ture style. His work portrays nature in harmo- nious, gentle colors that evoke calmness.
impressionism excellent depiction effectively arranged un- exrepresented explained
18


Reviews:
[{"review_id": "GPnaptGv2X", "reviewer": "Reviewer_Bti9", "summary": "This paper introduces a method to extract training data from personalized diffusion models (DMs). The method approximates the fine-tuned model's distribution as an interpolation between the pretrained model's distribution and the fine-tuning data distribution. By extrapolating the score functions of these models, the generation can be guided toward regions with high probability in the fine-tuning data distribution. Subsequently, clustering is used to identify the most likely matches to the original training data.\n\n## update after rebuttal\nI am satisfied with the rebuttal, adding experiments with FLUX.1 that show consistent improvements over the baseline method and showing how the extraction rates are correlated with training iterations, which shows the efficacy of the method follows model memorization.\n\nI am convinced to raise my score to Accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback!\n\n1. About Experiment Design and Analysis: \n\n> Experiments on model architectures only account for convolution-based diffusion. Current state-of-the-art generative models, like SD3, Flux or Sana, feature a transformer-based architecture. The paper would benefit from extending the evaluations to these.\n\nWe update new results using FLUX.1 [dev]. We experiment on Lora scenario with the official scripts provided by diffusers. The training iterations are fixed to 150$N_0$ as suggested by the repository and other hyper-parameters keep the same as the one in the repository. We compare our method with baseline (CFG), both methods show best performance when the guidance strength $w’$ is 3.0, where we can see the improvement is consistent. We will add these results in the revised paper.\n\n|                |   AS  | A-ESR($\\tau$=0.7) | A-ESR($\\tau$=0.6) |\n|:--------------:|:-----:|:-----------------:|:----------------:|\n| CFG+Clustering | 0.407 |       0.03        |       0.20       |\n|   FineXtract   | 0.496 |       0.30        |       0.43       |\n\n2. About Weakness1 and Question:\n\n> The extraction success rate is significantly better than the baselines, but still nearly 20% of the total, which seems limited.\n\n> Given that the extraction success rate is around 20% in most cases, what factors do you believe limit the extraction of the remaining training data? Would further improvements to the guidance or clustering components potentially increase this rate, or are there fundamental limitations to how much information can be extracted?\n\nWe present updated results demonstrating how the extraction success rate increases with the number of training iterations across four classes of the WikiArt dataset, using DreamBooth as the fine-tuning method on SD v1.4.\n\n| Training Iterations        | AS     | A-ESR($\\tau$=0.6) |\n|----------------------------|--------|-------------------|\n| 100$N_0$                 | 0.350  | 0.03              |\n| 150$N_0$                 | 0.360  | 0.05              |\n| 200$N_0$ (commonly used setting) | 0.501  | 0.35              |\n| 300$N_0$                | 0.564  | 0.58              |\n| 400$N_0$                 | 0.594  | 0.68              |\n\n\n\nThese results suggest that the extraction performance is primarily dependent on the extent of the information the model memorizes. Our current result, above 20% at most cases, is based on the default training configuration and the current checkpoint. We believe this limitation is largely due to the model's inherent memorization capacity.\n\nPotential improvements may arise from leveraging additional information within the model. Our current approach relies solely on the output of predicted noise, utilizing the model in an end-to-end manner. Future work could focus on further analysis, such as measuring attention responses to different inputs or optimizing input noise, which might enhance the method.\n\n3. About Weakness2:\n\n> The computational requirements of running two models simultaneously are higher than the baseline (more GPU memory), which may limit practical applicability on consumer-grade equipment.\n\nEven though we need to load two models onto the GPU simultaneously, this overhead is minimized because typically only a specific component is fine-tuned. For instance, in DreamBooth, the UNet is typically fine-tuned, while the text encoder remains unchanged. In LoRA, only the LoRA component is fine-tuned. Therefore, when using our method in such scenarios, the additional GPU memory usage is limited to the fine-tuned module. Moreover, FineXtract is an inference-only method and does not require the memory costs from gradient backpropagation. Therefore, loading two models does not limit practical applicability a lot on consumer-grade equipment in practice.\n\n4. Other comments or suggestions:\n\n> Additional visualizations showing the distribution shifts between pretrained and fine-tuned models could help readers better understand the theoretical foundations.\n\nThank you for your valuable advice. We will revise the paper accordingly.", "claims_and_evidence": "- The primary claim that fine-tuned diffusion models leak information about their training data is well-supported by the experimental results, and the method successfully extracts training images from various models\n- The theoretical claim about approximating the fine-tuned model's distribution as an interpolation between the pretrained model and the training data distribution is mathematically consistent and tested with effective practical results.\n- The performance claims are supported by experiments across different models, fine-tuning methods, and datasets, showing consistent improvements over baselines. Effectiveness is also demonstrated on real-world checkpoints from HuggingFace.", "methods_and_evaluation_criteria": "- The proposed method of model guidance through extrapolation is well-motivated and appropriate for the task, leveraging the mathematical relationship between score matching and diffusion models.\n- The evaluation metrics are appropriate and refer to previous work in the field.\n- The choice of datasets for style and object learning is appropriate and common for the use of personalized diffusion models.", "theoretical_claims": "I have read through Section 4 and the derivations appear sound. However, I haven't checked the correctness of them.", "experimental_designs_or_analyses": "- The ablation studies on guidance scale and correction term provide valuable insights into the sensitivity of the method, while experiments on real-world checkpoints are particularly valuable.\n- The defense experiments are also interesting, showing that they can reduce extraction success at the cost of generation quality, highlighting practical tradeoffs.\n- Experiments on model architectures only account for convolution-based diffusion. Current state-of-the-art generative models, like SD3, Flux or Sana, feature a transformer-based architecture. The paper would benefit from extending the evaluations to these.", "relation_to_broader_scientific_literature": "This work extends previous research on memorization and data extraction in diffusion models, focusing on personalized models. It also contributes to the area of privacy and copyright in GenAI by providing concrete evidence of data leakage risks.", "essential_references_not_discussed": "No missing essential related works to my knowledge.", "other_strengths_and_weaknesses": "### Strengths\n- The formulation of fine-tuned model distribution as an interpolation between the base model and training data distributions provides an elegant theoretical framework that could be applied to other problems beyond data extraction.\n- The work is original in its application of score matching and guidance to the problem of extracting private training data.\n\n### Weaknesses\n- The extraction success rate is significantly better than the baselines, but still nearly 20% of the total, which seems limited.\n- The computational requirements of running two models simultaneously are higher than the baseline (more GPU memory), which may limit practical applicability on consumer-grade equipment.", "other_comments_or_suggestions": "Additional visualizations showing the distribution shifts between pretrained and fine-tuned models could help readers better understand the theoretical foundations.", "questions_for_authors": "Given that the extraction success rate is around 20% in most cases, what factors do you believe limit the extraction of the remaining training data? Would further improvements to the guidance or clustering components potentially increase this rate, or are there fundamental limitations to how much information can be extracted?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "edftZf4qTx", "reviewer": "Reviewer_AHj2", "summary": "This paper introduces a novel technique for extracting fine-tuning data from personalized diffusion models, distinguishing it from prior work on data extraction in standard diffusion models. The additional constraints imposed by the fine-tuning phase have real-world implications, enabling more effective data extraction by leveraging the knowledge that the current model is a fine-tuned version of a base model.\n\nThe key insight is that the fine-tuned model's score prediction can be expressed as a weighted combination of the base model's score and the score of the fine-tuning dataset distribution. This formulation allows for an analytical conversion to isolate the score of the fine-tuning distribution, which can then be used for sampling and subsequent clustering to recover training data.\n\nThe proposed method is evaluated on a customized generation benchmark, demonstrating improved performance by leveraging this new perspective.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We greatly appreciate your positive comments!", "claims_and_evidence": "Yes, the claims are well validated.", "methods_and_evaluation_criteria": "The overall approach is well-founded. I appreciate the novel problem setup and the insightful method developed around it. The evaluation is robust and effectively supports the proposed technique.", "theoretical_claims": "Yes, I checked the equations for deriving the score for the distribution of the finetuned dataset", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "This paper presents a novel approach to improving training data extraction from pre-trained diffusion models by introducing a tailored guidance term that isolates the influence of fine-tuning. Given the widespread availability of fine-tuned models online, this technique has good practical relevance.", "essential_references_not_discussed": "no.", "other_strengths_and_weaknesses": "The overall illustration is clear. And the method makes intuitively sense and is easy to derive and implement.", "other_comments_or_suggestions": "not applicable.", "questions_for_authors": "not applicable.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DWd04w5OL9", "reviewer": "Reviewer_ovaG", "summary": "This paper introduces FineXtract, a framework for extracting data used in the fine-tuning of personalized diffusion models. The authors propose a parametric approach to approximate the fine-tuning data distribution by extrapolating the original output distributions of both the pre-trained and fine-tuned models. Subsequently, a clustering algorithm is applied to identify probable fine-tuning images from the generated samples. The proposed method is evaluated across various fine-tuning scenarios, including real-world cases, achieving an Extraction Success Rate of approximately 20%.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback!\n\n1. For Questions1:\n\n>  It is commendable that the paper discusses strategies for partially extracting and extending captions when they are unavailable. However, did the authors evaluate FineXtract using these extracted captions? If so, how does the performance compare to using ground-truth captions?\n\nAs the caption extraction algorithm is not the core contribution of this paper, we focus on the extraction with full captions in our main experiments, consistent with the settings of previous works [1-3].\n\nNonetheless, we admit that evaluating FineXtract using the extracted captions is highly valuable for fully assessing its effectiveness. Therefore, we update the results using the extracted captions (3 words in length) shown in Tab 5. We observe that, although the extraction success rate decreases compared to using full captions, it remains significantly higher than when no caption information is used. FineXtract, when applied to extracted captions, achieves a notably higher extraction success rate compared to the baseline. This confirms the effectiveness of both the proposed caption extraction approach and FineXtract.\n\n|                                 |   AS   | A-ESR($\\tau$=0.6) |\n|:-------------------------------:|:------:|:-----------------:|\n|    FineXtract (Full Caption)    | 0.501  |       0.35        |\n|  FineXtract (Extracted Caption) |  0.314 |        0.15       |\n|    FineXtract (Empty Caption)   |  0.192 |       0.00        |\n|        CFG (Full Caption)       |  0.434 |        0.23       |\n|     CFG (Extracted Caption)     |  0.308 |        0.08       |\n| Direct Text2img (Empty Caption) |  0.146 |       0.00        |\n\n[1]Carlini N, Hayes J, Nasr M, et al. Extracting training data from diffusion models[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 5253-5270.\n\n[2]Somepalli G, Singla V, Goldblum M, et al. Diffusion art or digital forgery? investigating data replication in diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 6048-6058.\n\n[3]Somepalli G, Singla V, Goldblum M, et al. Understanding and mitigating copying in diffusion models[J]. Advances in Neural Information Processing Systems, 2023, 36: 47783-47803.\n\n2. For Questions2:\n\n> Could the authors provide further discussion on the underlying principles or insights for identifying   or setting the grid search range? \n\n>  Furthermore, when the fine-tuning dataset and iteration count are fixed, does a larger optimal   suggest that the model or fine-tuning approach is more prone to overfitting or memorization of the training data?\n\nIntuitively, with longer training iterations, the distribution learned by the fine-tuned model should better align with the fine-tuned data distribution. In other words, $ p_{\\theta'}(x) $ should approach $ q(x) $ and become more distant from $ p_{\\theta'}(x) $ in Eq. (3). Therefore, the optimal $ \\lambda $ should increase (closer to 1), and the optimal $ w = \\frac{1}{\\lambda} $ should decrease with longer training iterations. Similiary, for conditional diffusion model in Eq. (7), the optimal $\\lambda'$ should also increase with the optimal $ w' = \\frac{1}{\\lambda'} $ decreases with longer training iterations. \n\n\nWe update the results using four classes of images from 4 classes of WikiArt under DreamBooth and perform a grid search to find the optimal hyperparameter $ w' $ under different training iterations. We set the fixed term $ k = 0 $ for simplicity, focusing only on $ w' $. The experiment results are available via an anonymous link: [Grid search for the best w' for different classes](https://drive.google.com/file/d/1HGMN1jv4x3jrMP21QgEa2Z9aYnB-4oSj/view?usp=sharing).\n\nOur findings show that, for our method, the optimal $ w' $ tends to decrease (the optimal $\\lambda'=\\frac{1}{w'}$ tends to increase) as training iterations increase in most cases. However, due to the inherent randomness in the clustering process, this trend is not consistent across all scenarios.\n\n\nWe also observe that when the AS is higher, indicating that the model has memorized more, the optimal $w' $ tends to be smaller ( the optimal $\\lambda'=\\frac{1}{w'}$ tends be larger) . The experiment results are available via an anonymous link: [Experiment result for AS at best $w'$](https://drive.google.com/file/d/1rMAXcjBODxSZ7U523_SOeYC7X0RJcIEN/view?usp=sharing).\n\n In other words, a larger $\\lambda'$ likely suggests that the model, or the fine-tuning approach, is more prone to overfitting or memorization of the training data, as noted by the reviewer.\n\nWe will include more extensive research on this aspect in the revised paper.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Nan", "experimental_designs_or_analyses": "Yes, I checked section 5 and all sections in the Appendix.", "relation_to_broader_scientific_literature": "Previous studies on data extraction have primarily focused on pre-trained diffusion models, while research on extracting data from personalized diffusion models—particularly few-shot fine-tuned models—remains scarce. This work introduces a novel approach by constructing the fine-tuning data distribution through extrapolation between the pre-trained and fine-tuned models. To the best of my knowledge, this is a new perspective that is likely to provide valuable insights to the research community.", "essential_references_not_discussed": "Nan", "other_strengths_and_weaknesses": "Strengths: The paper is overall well constructed and presented. The proposed approach is principled. The empirical study is thorough. \n\nWeaknesses: See questions.", "other_comments_or_suggestions": "Nan", "questions_for_authors": "I have two questions regarding the empirical study:\n\n1. Caption Accessibility Assumption: As the authors also noted, the assumption that training captions are fully accessible seems somewhat strong for practical scenarios. It is commendable that the paper discusses strategies for partially extracting and extending captions when they are unavailable. However, did the authors evaluate FineXtract using these extracted captions? If so, how does the performance compare to using ground-truth captions?\n\n2. Hyperparameter $\\lambda{\\prime}$  Selection: It appears that the key hyperparameter  $\\lambda{\\prime}$  is selected via grid search (among 1.0, 2.0, 3.0, 4.0, and 5.0), and its optimal value varies across different fine-tuning settings and data extraction approaches. Could the authors provide further discussion on the underlying principles or insights for identifying  $\\lambda{\\prime}$ or setting the grid search range? For example, I suspect that the optimal  $\\lambda{\\prime}$  depends on the number of fine-tuning iterations. Would it be possible to fix the fine-tuning dataset and the data extraction approach, and systematically analyze how the optimal  $\\lambda{\\prime}$  changes with the number of fine-tuning iterations? Furthermore, when the fine-tuning dataset and iteration count are fixed, does a larger optimal  $\\lambda{\\prime}$  suggest that the model or fine-tuning approach is more prone to overfitting or memorization of the training data?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IzyWEvgwZp", "reviewer": "Reviewer_ZxJM", "summary": "The paper proposes a model guidance to extract fine-tuning data, that leverages its base pre-trained model as a guidance. The proposed method “model guidance” can sample from the learned distribution of the fine-tuned models via simple guidance techniques. They further propose a new clustering algorithm for sampling within high-probability regions by constructing an image graph. Experiments on various datasets confirm its effectiveness, supported by ablation studies, and real-world applications, such as using checkpoints from the community.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback!\n\n\n1. For Weakness1:\n\n>  I couldn't find the training data extraction result using the extracted prompt. In my opinion, this part is crucial for highlighting the paper's practical motivation, so the results should be reported.\n\nOur default setting uses captions as accessible, aligning with previous works [1-3]. We observe that captions are accessible in many cases, particularly on real checkpoints on Civitai. This is especially evident for DreamBooth, where several special tokens within the training captions (such as \"a sks dog\") are always available to ensure the correct application of the models.\n\nNonetheless, there are scenarios where captions are not accessible, and we admit that data extraction result using the extracted prompt is helpful. Therefore, we update the results using the extracted captions (3 words in length) shown in Tab 5. \n\n\n|                                 |   AS   | A-ESR($\\tau$=0.6) |\n|:-------------------------------:|:------:|:-----------------:|\n|    FineXtract (Full Caption)    | 0.501  |       0.35        |\n|  FineXtract (Extracted Caption) |  0.314 |        0.15       |\n|    FineXtract (Empty Caption)   |  0.192 |       0.00        |\n|        CFG (Full Caption)       |  0.434 |        0.23       |\n|     CFG (Extracted Caption)     |  0.308 |        0.08       |\n| Direct Text2img (Empty Caption) |  0.146 |       0.00        |\n\n\nWe observe that though the extraction success rate decreases compared with extraction with full captions,  it is significantly stronger than performing extraction without any caption information. It confirms the effectiveness of the proposed caption extraction approach. Additionally, FineXtract achieves notably higher extraction performance under extracted captions compared to the baseline.\n\n[1]Carlini N, Hayes J, Nasr M, et al. Extracting training data from diffusion models[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 5253-5270.\n\n[2]Somepalli G, Singla V, Goldblum M, et al. Diffusion art or digital forgery? investigating data replication in diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 6048-6058.\n\n[3]Somepalli G, Singla V, Goldblum M, et al. Understanding and mitigating copying in diffusion models[J]. Advances in Neural Information Processing Systems, 2023, 36: 47783-47803.\n\n\n\n2. For Weakness2:\n\n> An ablation study on the clustering method is essential, and further explanations, such as qualitative examples, would be helpful.\n\nWithout the clustering method, the attacker can only generate a number of data points but cannot determine which ones could lead to a successful extraction. This is reflected in our ablation study in Fig 4(b), where only 1 generated image per training image is used. The result is presented again in the following table to emphasize how the clustering component significantly improves the AS and the A-ESR.\n\n|                                 |   AS   | A-ESR($\\tau$=0.7) | A-ESR($\\tau$=0.6) |\n|:-------------------------------:|:------:|:-----------------:|:-----------------:|\n|     CFG (without Clustering)    | 0.280  |       0.00        |       0.10        |\n| FineXtract (without Clustering) | 0.338  |       0.05        |       0.13        |\n|      CFG (with Clustering)      | 0.434  |       0.08        |       0.23        |\n|   FineXtract (with Clustering)  | 0.501  |       0.15        |       0.35        |\n\n3. For Questions: \n\n> How does the method perform on recent models, such as FLUX? \n\nWe update the results using FLUX.1 [dev]. The DreamBooth fine-tuning on FLUX.1 [dev] requires more than 100GB per GPU, which we are currently unable to run. However, we have conducted experiments in the LoRA scenario using the official scripts provided by Diffusers. The training iterations are fixed at 150$N_0$, as suggested by the repository, and other hyperparameters remain the same as those in the repo. We compare our method with the baseline (CFG), and both methods show the best performance when the guidance strength $w’$ is set to 3.0, with the improvement being consistent.\n\n|                |   AS  | A-ESR($\\tau$=0.7) | A-ESR($\\tau$=0.6) |\n|:--------------:|:-----:|:-----------------:|:----------------:|\n| CFG+Clustering | 0.407 |       0.03        |       0.20       |\n|   FineXtract   | 0.496 |       0.30        |       0.43       |", "claims_and_evidence": "Yes, the paper’s claims are supported by experiments. But some parts are missing; see Weaknesses for details.", "methods_and_evaluation_criteria": "Yes, the paper’s methods are supported by experiments.", "theoretical_claims": "Yes, the paper’s equations are reasonable.", "experimental_designs_or_analyses": "Yes, the paper’s methods are supported by experiments. But some parts are missing; see Weaknesses for details.", "relation_to_broader_scientific_literature": "This work is the first try to leverage a base pre-trained model to extract its fine-tuning dataset. Most existing works did not distinguish this from pre-training.", "essential_references_not_discussed": "No, to the best of my knowledge, there are no related works that should be further compared or discussed.", "other_strengths_and_weaknesses": "**Sterngth** \n\n- Model guidance is straightforward and aligns well with the current trend of using the same base model in real-world applications.\n- The paper is easy to read and follow.\n- Experiments are provided on various models and concepts, and the experiments using checkpoints from communities like Hugging Face were particularly good.\n\n\n**Weakness**\n\n- The paper assumes that the training caption is given. Although I found the training caption extraction part in the Appendix, I couldn't find the training data extraction result using the extracted prompt. In my opinion, this part is crucial for highlighting the paper's practical motivation, so the results should be reported.\n- The model guidance method sounds reasonable, but the clustering approach is not very intuitive and novel. An ablation study on the clustering method is essential, and further explanations, such as qualitative examples, would be helpful.", "other_comments_or_suggestions": "Please refer to Questions or Weaknesses.", "questions_for_authors": "How does the method perform on recent models, such as FLUX? In the case of DreamBooth, better models tend to show higher generalizability rather than memorization. I would like to see results related to this.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel method for extracting training data from personalized diffusion models. To achieve this, the authors propose model guidance and clustering techniques.

Strengths

* Simple yet effective method
* Addresses an important research topic with real-world relevance
* Demonstrates results across a variety of models

Weaknesses
* Relies on assumptions about training captions

The authors have addressed the reviewers’ concerns in the rebuttal, and I believe this paper makes a meaningful contribution that will be valuable to the community.
Decision: Accept (poster)