Image Copy Detection for Diffusion Models
Authors: Wenhao Wang
Yifan Sun
Zhentao Tan
Yi Yang
Venue: NeurIPS 2024 poster (2024)
Keywords: Image Copy Detection
Diffusion Models
Replicated Content
Primary area: machine_vision
TL;DR: We propose a timely and important ICD task, i.e, Image Copy Detection for Diffusion Models (ICDiff), designed specifically to identify the replication caused by diffusion models.

Abstract: Images produced by diffusion models are increasingly popular in digital artwork and visual marketing. However, such generated images might replicate content from existing ones and pose the challenge of content originality. Existing Image Copy Detection (ICD) models, though accurate in detecting hand-crafted replicas, overlook the challenge from diffusion models. This motivates us to introduce ICDiff, the first ICD specialized for diffusion models. To this end, we construct a Diffusion-Replication (D-Rep) dataset and correspondingly propose a novel deep embedding method. D-Rep uses a state-of-the-art diffusion model (Stable Diffusion V1.5) to generate 40, 000 image-replica pairs, which are manually annotated into 6 replication levels ranging from 0 (no replication) to 5 (total replication). Our method, PDF-Embedding, transforms the replication level of each image-replica pair into a probability density function (PDF) as the supervision signal. The intuition is that the probability of neighboring replication levels should be continuous and smooth. Experimental results show that PDF-Embedding surpasses protocol-driven methods and non-PDF choices on the D-Rep test set. Moreover, by utilizing PDF-Embedding, we find that the replication ratios of well-known diffusion models against an open-source gallery range from 10% to 20%. The project is publicly available at https://icdiff.github.io/.

Image Copy Detection for Diffusion Models
Wenhao Wang', Yifan Sun?; Zhentao Tan', Yi Yang" University of Technology Sydney Baidu Inc. *Zhejiang University
Midjourney
New Bing
DALL-E
SDXL
DeepFloyd IF
GLIDE
Generated
Matched
Figure 1: Some generated images (top) from diffusion models replicates the contents of existing images (bottom). The existing (matched) images are from LAION-Aesthetic [1]. The diffusion models include both commercial and open-source ones.
Abstract
Images produced by diffusion models are increasingly popular in digital artwork and visual marketing. However, such generated images might replicate content from existing ones and pose the challenge of content originality. Existing Image Copy Detection (ICD) models, though accurate detecting hand-crafted replicas, overlook the challenge from diffusion models. This motivates us to introduce ICDiff, the first ICD specialized for diffusion models. To this end, we construct a Inffusion-RRplicattion (D-Rep) dataset and correspondingly propose novel deep embedding method. D-Rep uses a state-of-the-an diffusion model (Stable Diffu- sion V1.5) to generate 40, 000 image-replica pairs, which manually annotated into 6 replication levels ranging from 0 (no replication) to (total replication). Our method, PDF-Embedding, transforms the replication level of each image- replica pair into a probability density function (PDF) as the supervision signal. The intuition is that the probability of neighboring replication levels should be continuous and smooth. Experimental results show that PDF-Embedding surpasses protocol-driven methods and non-PDF choices on the D-Rep test set. Moreover, by utilizing PDF-Embedding, we find that the replication ratios of well-known diffusion models against an open-source gallery range from 10% to 20%. The project is publicly available at https: //icdif/ ..f..giii thub. io/
Introduction
Diffusion models have gained popularity due to their ability to generate high-quality images. phenomenon accompanying this trend is that these generated images might replicate content from
"Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Current ICD
ICD for Diffusion Models
Original image
Replicated by transformations
Replicated by Stoble Diffusion
Figure The comparison between current ICD with the ICDiff. The current ICD focuses on detecting edited copies generated by transformations like horizontal flips, random rotations, and random crops. In contrast, the ICDiff aims to detect replication generated by diffusion models, such as Stable Diffusion [2]. (Source of the original image: Lawsuit from Getty Images.)
existing ones. In Fig. I, we choose six well-known diffusion models [3, 4, 5, 6, 7, 8] to illustrate this replication phenomenon. The content replication is acceptable for some (fair) use while interest holders may regard others as copyright infringement [9, 10, 11]. This paper leaves this dispute alone, and focuses scientific problem: How identify the content replication brought by diffusion models?
Image Copy Detection (ICD) provides general solution the above demand: it identifies whether an image is copied from a reference gallery after being tampered with. However, the current ICD methods are trained using hand-crafted image transformations (e.g.. horizontal flips, random rotations, and random crops) and overlook the challenge from diffusion models. Empirically, we find existing ICD methods can be easily confused by diffusion generrttd replicas (as detailed in Table 3). We infer it is because the tamper patterns underlying diffusion-generns gennrratd replicas (Fig. 2 right) are different from hand-crafted ones (Fig. 2 middle), yielding a considerable pattern gap.
In this paper, we introduce ICDiff, the first ICD specialized for diffusion- -ennerrtee replicas. Our efforts mainly involve building a new ICD dataset and proposing novel deep embedding method.
A Diffusion Replication (D-Rep) dataset. D-Rep consists of 40, 000 image-replica pairs, in which each replica is generated by diffusion model. Specifically, the images from LAION-Aesthetic V2 [1]. while their replicas are generated by Stable Diffusion V1.5 [12]. To make the replica generation more efficient, we search out the text prompts (from DiffusionDB [13]t are similar to the titles of Aesthetii V2 images, input these text prompts into Stable Diffusion V1.5, and generate many redundant candidate replicas. Given these candidate replicas, we employ human annotators to label the replication level of each generated image against a corresponding LAION-Aesthen The annotation results in 40, 000 image-replica pairs with 6 replication levels ranging from 0 (no replication) to 5 (total replication). We divide D-Rep into a training set with 90% (36, 000) pairs and a test set with the remaining 10% (4, 000) pairs.
image.
A novel method named DDF-Embedding, The ICD methods rely on deep embedding learning at their core. In the deep embedding space, the replica should be close to its original image and far away from other images. Compared with popular deep embedding methods, our PDbeddidd learns Probability- DabbabilityDDenitt---e Density-Functionn between two images, instead of a similarity score. More concretely, PDF-Embedding transforms the replication level of each image-replica pair into a PDF as the supervision signal. The intuition is that the probability of neighboring replication levels should be continuous and smooth. For instance, if an image-replica pair is annotated as level-3 replication, the probabilities for level-2 and level-4 replications should also not be significantly low.
PDF-Embedding predicts the probability scores on all the replication levels simultaneously in two steps: 1) extracting 6 feature vectors in parallel from both the real image and its replica, respectively and 2) calculating inner products (between two images) to indicate the probability score at 6 corresponding replication levels. The largest-scored entry indicates the predicted replication level. Experimentally, we prove the effectiveness of our method by comparing it with popular deep embedding models and protocol-driven methods trained on our D-Rep. Moreover, we evaluate the replication of six famous diffusion models and provide a comprehensive analysis.
In conclusion, our key contributions are as follows:

---

1. We propose a timely and important ICD task. i.e, Image Copy Detection for Diffusion Models (ICDift), designed specifically to identify the replication caused by diffusion models.
2. We build the first ICDiff dataset and introduce PDF-Embeee Embeddingg PDF- Embeedinn transforms replication levels into probability density functions (PDFs) and learns set of representative vectors for each image.
baseline method.
3. Extensive experimental results demonstrate the efficiency of our proposed method. Moreover, we discover that between 10% to 20% of images generated by six well-known diffusion models replicate contents of a large-scale image gallery.
2 Related Works
2.1 Existing Image Copy Detection Methods
Current ICD methods try to detect replications by learning the invariance of image transformations. For example, ASL. [14] considers the relationship between image transformations and hard negative samples. AnyPattern [20] and PE-ICD [21] build benchmarks and propose solutions that focus on novel patterns in real nol-world scenarios. SSCD [15] reveals that self-suuervised contrastive training inherently relies on image transformations, and thus adapts InfoNCE [16] by introducing a differential entropy regularization. BoT [17] incorporates approximately ten types of image transformations, combined with various tricks, to train an ICD model. By increasing the intensity of transformations gradually. CNNCL [18] successfully detects hard positives using a simple contrastive loss and memory bank. EfNet [19] ensembles models trained with different image transformations to boost the performance. In this paper, we discover that capturing the invariance of image transformations is ineffective for detecting copies generated by diffusion models. Consequently, we manually label new dataset and train a specialized ICD model.
2.2 Replication Diffusion Models
Past research has explored the replication problems associated with diffusion models. The study by [10] questions if diffusion models generate unique artworks or simply mirror the content from their training data. Research teams from Google, highlighted in [22], note that diffusion models reveal their training data during content generation. Other studies prevent generating replications from the perspectives of both training diffusion models [23, 24, 25, 26, 27] and copyright holders [28. 29, 30]. Some experts, such as those in [24], find that the replication of data within training sets might be a significant factor leading to copying behaviors in diffusion models. To address this, [31] proposes an algorithmic chain to de-duplicate the training sources like LAION-2B [1]. In contrast to these efforts, our ICDiff offers unique perspective. Specifically, unlike those that directly using existing image descriptors (such as those from CLIP [32] and SSCD [15]), we manually-label a dataset and develop specialized ICD algorithm. By implementing our method, the analytical tools and preventative strategies proposed in existing studies may achieve greater efficacy.
3 Benchmark
This section introduces the proposed ICD for diffusion models (ICDiff), including our dataset (D-Rep) and the corresponding evaluation protocols.
3.1 D-Rep Dataset
Current ICD [33, 34, 35, 14, 21, 20] primarily focuses on the replica challenges brought by hand- crafted transformations. In contrast, our ICDiff aims to address the replication issues caused by diffusion models [3, 5, 6. 7, 8]. To facilitate ICDiff research, we construct D-Rep dataset, which characterized for diffusion based replication (See Fig. 3 and the Appendix (Section A) for the examples of diffusion based replica). The construction process involves generating candidate pairs followed by manual labeling.
Generating candidate pairs. It consists of (1) selecting the top 40, 000 most similar prompts and titles: this selection provides an abundant image-replica pair source. In detail, we use the Sentence Transformer [36] to encode the 1.8 million real-user generated prompts from DiffusionDB [13] and
3

---

Level 5
6.94%
Level
17.6
25.54
Leve
Level 2
18.009
15.990%
Level
Level 15-92%
Figure 3: The demonstration of the manual-labeled D-Rep dataset. The percentages on the left show the proportion of images with particular level.
the 12 million image titles from LAION-Aesthetie V2 6+ [1], and then utilize the computed cosine similarities to compare; (2) obtaining the candidate pairs: the generated images are produced using the prompts with Stable Diffusion V1.5 [12], and the real images are fetched based on the titles.
Manual labeling. We generally follow the definition of replication in [10] and further define six levels of replication (0 to 5). higher level indicates a greater degree that the generated image replicates the real image. Due to the complex nature of diffusion-generann
images, we use multiple datasets as shown in their Fig. 2. We then train ten professional labelers assign these levels to the 40, 000 candidate pairs: Initially, we assign 4, 000 image pairs to each labeler. labelers are confident in their judgment of image pair, they will directly assign label. Otherwise, they will place the image pair in undecided pool. On average, each labeler has about 600 undecided pairs. Finally, for each undecided pair, we vote reach final decision. For example, if the votes for undecided pair are 2, 2, 3, 3, 3, 3. 3, 4, 4, the final label assigned 3. Given the complexity of this labeling task, it took both the labelers and our team one month to finish the process. To maintain authenticity, we did not pre-determine the proportion of each score. The resulting proportions are on the left side of Fig. 3.
levels instead of the binary levels used in [10]. which employed
3.2 Evaluation Protocols
To evaluate ICD models the D-Rep dataset, we divide the dataset into 90/10 training/test split and design two evaluation protocols: Pearson Correlation Coefficient (PCC) and Relative Deviation (RD).
Pearson Correlation Coefficient (PCC). The PCC is a measure used to quantify the linear relation- ship between two sequences. When PCC is near 11, it indicates strong positive negative relationship. PCC near 0, there's little to no correlation between the sequences. Herein, we consider two sequences, the predicted replication level s (5f sy ...,,., .....,,, s.)) and the ground-truth
s' (s), s2...., (n is the number of test pairs). The PCC for ICDiff is defined as:
Σ
8D) (o)
=
PCC
mml
(1)
50)² x
Σ
{ 5²)
where SP and g are the mean values of s and s respectively.

---

Transformer blocks
Ground truth level
Transformer blocks
PDF converted
ground truth
KL divergence
Predicted PDF
Dot products
Class tokens
Patch tokens
Feature vectors of replica
Feature vectors real image
Figure 4: The demonstration of the proposed Embeddddd.. Initially, PDF-Embedding converts manually-labell replication levels into probability density functions (PDFs). To learn from these PDFs, we use set of vectors as the representation of an image.
limitation of the PCC is its insensitivity global shifts. If all the predictions differ from their corresponding ground truth with the same shift, the PCC does not reflect such a shift and remains large. To overcome this limitation, we propose a new metric called the Relative Deviation (RD).
Relative Deviation (RD). We use RD to quantify the normalized deviation between the predicted and the labeled levels. By normalizing against the maximum possible deviation, RD provides measure of how close the predictions are to the labeled levels on a scale of O to 1. The RD is calculated by
RD
max (N
s st
where N is the highest replication level in our D-Rep.
The Preference for RD over Absolute One. Here we show the preference for employing RD over the absolute one through two illustrative examples. We denote the relative and absolute deviation of the ith test pair as: S. max
- st."!' and T, -
(1) For a sample with s 3. if s 3. both S, and T; equal O; however, if s 0 (representing the worst prediction),
1 and 0.6. Here, S, adjusts the worst prediction value of 1.
(2) the first scenario, where of 3 and s. = 0, St = and T; 0.6. In the second scenario, where of 5 and s = 2, S; 0.6 and T; 0.6. For both cases, T; remains the same at 0.6, whereas S, values differ. Nevertheless, the two scenarios cannot deteriorate further; in the second, can. The value of S, accurately captures this distinction, whereas T, does not.
distinct: in the first, the prediction
4 Method
This section introduces our proposed PDF-Edddd for ICDiff. Embeddind converts each replication level into a probability density function (PDF). To facilitate learning from these PDFs, we expand the original representative vector into a set of vectors. The demonstration of PDF-Embedding is displayed in Fig. 4.
4.1 Converting Level to Probability Density
Given replication level s s', we first normalize it into pf s² /max(s!). Then transfer p into PDF denoted as g(x) where E [0, 1] indicates each possible normalized level. The
Although we acknowledge that the random variables in this context are discrete, we still utilize the term "PDF" to effectively communicate intuition and present our method under ideal conditions.
5

---

intuition is that the probability distribution of neighboring replication levels should be continuous and ensuring that g(x) is a valid PDF; (2) g(x) 2 O, indicating the non-negativity of the PDF; and (3) g(x) S g('),
smooth. The function ((x) must satisfy the following conditions: (1) Su ((:)dz=
which implies that the density is maximized the normalized level p/. We use three different implementations for g(r):
Gaussian:
g0 A, - exp (- (x (a- 11²
(3)
2 0²
linear:
g(x A, 3) B. 11 pt/
(4)
and exponential:
g(x A. A) A exxp
(- X
ml),
(5)
where: the amplitude, and y the center; the standard deviation of Gaussian function, the slope of linear function, and À is the spread of exponential function.
The performance achieved with various converted PDFs is illustrated the experimental section. For additional details. please see the Appendix (Section B), which includes (1) the methodology for calculating distribution values, (2) the visualization of the learned distributions corresponding
to different image pairs, and (3) an analysis of the deviation rate from peak values.
4.2 Representing Image Set of Vectors
To facilitate learning from the converted PDFs, we utilize Vision Transformer (ViT) [37] to represent an image as set of vectors. Let's denote the patch tokens from a real image as X and from generated image as X the ViT model as f. and the number of layers in ViT as L. The feed-forward process can be expressed as:
[C, x ([C", x [C] [ / ([Co, xx))
(6)
where C is set of class tokens; C.F. is set of representative vectors for the real images, consisting of vectors ct.r: of......, of,,, and C is a set of representative vectors for the generated images, consisting of vectors o of. ........ Nigi N is the highest replication level.
Therefore, we can use another PDF la (x) (a E [0, 1J) to describe the predicted replication between two images by
Mxx C C.=y
which expands to:
((x)) = [cb,rr cb.g)) ) (ot,rs ot.g) ..... ,..., (cker, oh.gg]
(8)
where ( denotes the cosine similarity.
For training, recalling the PDF g(z) derived from the level, we define the final loss using the Kullback-Leibler (KI.) divergence:
g(x) h(x))
L = DKc(gllh) =
g(x) log
dx,
which serves as a measure of the disparity between two probability distributions. Additionally, in the Appendix (Section C), we demonstrate what the network captures during its learning phase.
During testing, the normalized level between two images is denoted by s satisfying h(x) < h(p'). As illustrated in Eqn. 8, h(x)) in practice is discrete within the interval [0, 1]. Consequently, the resulting level
j argmax h(x).
(10)
and the normalized level is quantified as +

---

Table The performance of publicly available models and our PDF-dddeeding on the D-Rep. For qualitative results, please refer to Section E in the Appendix.
Class
Method
PCC (%))
(%)
SLIP [41]
31.8
19.7
Vision-
BLIP [42]
34.8
41.6
language
ZeroVL [43]
36.3
36.5
Models
CLIP [32]
36.8
35.8
GPT-4V [44]
47.3
38.7
SimCLR [45]
7.2
49.4
Self-
MAE [46]
20.7
67.6
supervised
SimSiam [47]
33.5
45.4
Learning
MoCov3 [48]
35.7
40.3
Models
DINOv2 [49]
39.0
32.9
EfficientNet [50]
24.0
59.3
Supervised
Swin-B [51]
32.5
38.4
Pre-trained
ConvNeXt [52]
33.8
36.0
Models
DeiT-B [40]
35.3
41.7
ResNet-50 [53]
37.5
34.5
ASL [14]
5.6
78.1
Current ICD Models
CNNCL [18]
19.1
51.7
SSCD [15]
29.1
62.3
EfNet [19]
30.5
62.8
BoT [17]
35.6
53.8
Experiments
5.1 Training Details
We implement our PDF-Embedding using Py Torch [38] and distribute its training over 8 A100 GPUS. The ViT-B/16 [37] serves as the backbone and is pre-trained on the ImageNet dataset [39] using DeîT [40], unless specified otherwise. We resize images resolution of 224 224 before training. batch size of 512 is used, and the total training epochs is 25 with a cosine-dectee deaine-eereeaing learning rate.
5.2 Challenge from the ICDiff Task
This section benchmarks popular public models D-Rep test dataset. As Table 3 shows, conduct experiments extensively on vision-language models, self-supervised models, supervised pre- trained models, and current ICD models. We employ these models feature extractors and calculate the cosine similarity between pairs of image features (except for GPT-4V Turbo [44], see Section D in the Appendix for the implementation of it). For the computation of PCC and RD, we adjust the granularity by scaling the computed cosine similarities by factor of N. In the Appendix (Section E), we further present the concrete similarities predicted by these models and provide corresponding analysis. We observe that: (1) the large multimodal model GPT-4V Turbo [44] performs best in PCC, while the self-supervised model DINOv2 [49] excels in RD. This can be attributed to their pre-training on large, curated, and diverse dataset. Nevertheless, their performance remains somewhat limited, achieving only 47. 47.3% in PCC and 32.9% in RD. This underscores that even the best publicly available models have yet to effectively address the ICDiff task. (2) Current ICD models, like SSCD [15], which are referenced in analysis papers [10, 24] discussing the replication issues diffusion models, indeed show poor performance. For instance, SSCD [15] registers only 29. 19.1% in PCC and 62.3% RD. Even more advanced model, BoT [17], only manages 35.62 5..6% in PCC and 53.8% RD. These results underscore the need for specialized ICD method for diffusion models. Adopting specialized ICD approach will make their subsequent analysis more accurate and convincing. (3) Beyond these models, also observe that others underperform on the ICDiff task. This further emphasizes the necessity of training specialized ICDiff model.
5.3 The Effectiveness of PDF-Embedding
This section demonstrates the effectiveness of our proposed PDF-Embedding by (1) contrasting it against protocol-drive methods and non-PDF choices on the D-Rep dataset, (2) comparing between different distributions, and (3) comparing with other models in generalization settings.

---

Table 2: Our method demonstrates performance superiority others.
Method
PCC (%)
RD (%) +
Train (s/iler) +
Infer (s/ (s/img))
Match (s (s/pair)
Enlarging PCC
54.4
40.1
0.293
Reducing RD
15.1
29.9
0.294
2.02 x 10
1.02 x 10
Regression
40.3
28.1
0.292
One-hot Label
37.6
43.3
0.310
2.07 x 10
6.97 10-9
Label Smoothing
35.0
36.1
Ours (Gaussian)
53.7
24.0
Ours (Lincar)
54.0
24.6
0.310
2.07 10
6.97 10-9
Ours (Exp.)
56.3
25.6
56
31
56
11
56
54
30
54
10
54
30
29
29
29
52
2
50
POC
27
27
25
44
24
24
0.18
0.20
0.22
0.24
0.18
0.20
0.22
0.24
0.26
0.5
1.0
1.5
2.0
Figure 5: The comparison of different PIFFs Gaussian (left), linear (middle), and exponential (right). 4" is the amplitude in each PDF function (Eqn. to Eqn. 5).
Comparison against protocol-driven methods. Since we employ PCC and RD as the evaluation protocols, natural embedding learning would be directly using these protocols as the optimization objective, i.e., enlarging PCC and reducing RD. Moreover, we add another variant of "reducing RD", i.e., reducing the absolute deviation s - in a regression manner. The comparisons are summarized in Table 2, from which we draw three observations below: (1) Training on D-Rep with the protocol-drive driven method achieves good results on their specified protocol but performs bad for the other. While "Enlarging PCC" attains a commendable PCC, its RD of 40.1% indicates large deviation from ground truth. "Reducing RD" or "Reducing Deviation" shows relatively good RD (28.1%% 1%); however, they exhibit small PCC values that indicate low linear consistency. (2) Our proposed End--Embd surpasses these protocol- driven methods both PCC and RD. Compared against "Enlarging PCC", method improves PCC by 1.6% and decreases RD by 16.1%. Besides, our method achieves +16.0% PCC and -4.1%
1% RD compared against "Reducing RD" and "Reducing Deviation". (3) The computational overhead introduced by our method is negligible. First, compared to other options, our method only increases the training time by 5.8%. Second, our method introduces minimal additional inference time. Third, while method requires longer matching time, its magnitude is close to 10 which is negligible when compared to the inference time's 's magnitude of 10 Further discussions on the matching time in real-world scenarios can be found in Section 5.4.
Comparison against two non-PDF methods. Table 2, we also show the experimental results of our method under two standard supervising signals, i.e., "One-hot Label" and "Label Smoothing (e (e= (.50. In comparison, our PDF-Embedding using PDFs gains significant superiority, e.g., using exponential PDF is better than label smoothing by +21.3% PCC and 10.5% RD. This superiority validates our intuition that neighboring replication levels should be continuous and smooth.
Comparison between different PDF implementations. We compare between three different PDF implementations for the proposed PDb-Endedding Fig. 5. We observe that: (1) The exponential (convex) function benefits the PCC metric, whereas the Gaussian (concave) function favors the RD metric. The performance the linear function, which lacks curvature, falls between that of convex and concave functions. (2) Our method demonstrates robust performance across various distributions, reducing the challenge of selecting an optimal parameter. For example, when using the exponential function, the performance remains high when A ranges from 0.6 to 1.8. (3) A model

---

Table 3: The experiments for "Generalizability to other datasets diffusion models". The gray color
indicates training and testing on the images generated by the same diffusion model.
Midjo-
DAL-
DeepFl-
New
Class
Method
SD1.5
SDXL
GLIDE
urney
L-E 2
oyd IF
Bing
ST.IP [41]
0.685
0.680
0.668
0.710
0.688
0.718
0.699
Vision-
BLIP [42]
0.703
0.674
0.673
0.696
0.696
0.717
0.689
language
ZeroVL [43]
0.578
0.581
0.585
0.681
0.589
0.677
0.707
Models
CLIP [32]
0.646
0.665
0.694
0.728
0.695
0.735
0.727
GPT-4V [44]
0.661
0.655
0.705
0.731
0.732
0.747
0.744
SimCLR [45]
0.633
0.640
0.644
0.656
0.649
0.651
0.655
Self-
MAE [46]
0.489
0.488
0.487
0.492
0.487
0.489
0.490
supervised
SimSiam [47]
0.572
0.611
0.619
0.684
0.620
0.645
0.683
Learning
MoCov3 [48]
0.585
0.526
0.535
0.579
0.541
0.554
0.599
Models
DINOv2 [49]
0.766
0.529
0,593
0.723
0.652
0.734
0.751
EfficientNet [50]
0.116
0.185
0.215
0.241
0.171
0.210
0.268
Supervised
Swin-B [51]
0.334
0,387
0.391
0.514
0,409
0,430
0.561
Preitrd
ConvNeXt [52]
0.380
0.429
0,432
0.543
0,433
0.488
0.580
Models
DeïT-B [40]
0,386
0.478
0.496
0.603
0.528
0.525
0.694
ResNet-50 [53]
0.362
0.436
0.465
0.564
0.450
0.522
0.540
ASL
0.183
0.231
0.093
0.122
0.048
0.049
0,436
Current ICD
CNNCL [18]
0.201
0.311
0.270
0.347
0.279
0.358
0.349
SSCD [15]
0.116
0.181
0.180
0.303
0.166
0.239
0.266
Models
EfNet [19]
0.133
0.265
0.267
0.438
0.249
0.340
0.349
BoT [17]
0.216
0.345
0.346
0,477
0.338
0.401
0.489
Enlarging PCC
0.598
0.5
0.523
0.595
0.506
0.554
0.592
Models
Reducing RD
0.795
0.736
0.736
0.768
0.729
0.768
0.785
Trained
Regression
0.750
0.694
0.705
0.739
0.704
0.721
0.744
on D-Rep
Onc-hot Label
0.630
0.376
0,400
0.562
0.500
0.548
0.210
Label Smoothing
0.712
0.568
0,636
0.628
0.680
0.676
0.548
Gaussian PDF
0.787
0.754
0.784
0.774
0.774
0.780
0.776
Ours
Lincar PDF
0.822
0.758
0.798
0.794
0.782
0.794
0.790
Exponential PDF
0.831
0.814
0.826
0.804
0.802
0.818
0.794
supervised by smooth PDF outperforms that supervised by a steeper one (also see the corresponding distributions in Fig. 15 of the Appendix). That consists with our intuition again.
Our model has good generalizability compared all other methods. Because the collection process of the images from some diffusion models (see Appendix F) differs from the process used build the test set of our D-Rep dataset, is difficult label levels for them and the proposed PCC and RD are not suitable. In the Table 3, we consider a quantitative evaluation protocol that measures the average similarity predicted by a model for given N image pairs, which are manually labeled with the highest level. When normalized predictions. This setting practical because, the real world, most people's 's concerns focus on where replication indeed occurs. We manually confirm 100 such pairs for each diffusion model. We draw three conclusions: (1) Our PDF-Ennn EDDe-dddddiin is more generalizable compared to all zero-shot solutions, such CLIP, GPT4-V, and DINOv2; (2) Our PDF-Embedie
range of 0 to 1, larger value implies better
still surpasses all other plausible methods trained on the D-Rep dataset in the generalization setting: (3) Compared against testing on SD1.5 (same domain). for the proposed PDF- Embeddin,,,, there is significant performance drop the generalization setting.
5.4 Simulated Evaluation of Diffusion Models
In this section, we simulate a scenario using our trained Embedding to evaluate popular diffusion models. We select famous diffusion models, of which three are commercial, and another three are open source (See Section
V2 6+ dataset [1] as the gallery and investigate whether popular diffusion models replicate it. When assessing the replication ratio of diffusion models, we consider image pairs rated at Level and Level 5 to be replications.
the Appendix for more details). use the LAION-Aesther

---

[a) Midjourney
(b) New Bing 88.8999
20 20:2199
(c) DALL-E
17.70%
(d) SDXL
10.91%
HOOBEA
(e) DeepFloyd IF
17.52%
(0 GLIDE
16.24%
Figure Left: Examples of diffusion-barn replication fetched by our PDF-Embedding. The accompanying percentages indicate the replication ratio of each model. Right: Examples filtered by SSCD [15] in [10]. Compared to them, our results are more diverse: For example, the "Groot" generated by SDXL includes the whole body, whereas the original one features only the face; and the "Moai statues" created by DeepFloyd IF are positioned differently compared to the original image.
Evaluation results. Visualizations of matched examples and the replication ratios are shown in Fig. 6 (Left). For more visualizations, please refer to the Appendix (Section G). We observe that the replication ratios of these diffusion models roughly range between 10% and 20%. The most "aggressive" model is Midjourney [3] with a rate of 20.21%, %, whereas the "conservative" model is SDXL [6] 10.91%. We also include an analysis of failure cases in the Appendix (Section H).
Efficiency analysis. Efficiency crucial real-world scenarios. replication check might slow down the image generation speed of diffusion models. Our PDF- PDF-Emmedding requires only 2.07 10-3 seconds for inference and an additional 8.36 x 10 seconds for matching when comparing a generated image against reference dataset of 12 million images using standard A100 GPU. This time overhead is negligible compared to the time required for generating (several seconds).
Intuitive comparison with another ICD model. In [10], SSCD [15] is used feature extractor to identify replication, as illustrated in Fig. 6 (Right). In comparison, our PDF-Emmedding detects higher number of challenging cases ("hard positives"). Despite visual discrepancies between the generated and original images, replication has indeed occurred.
Conclusion
This paper investigates a particular and critical Image Copy Detection (ICD) problem: Image Copy Detection for Diffusion Models (ICDiff). We introduce the first ICDiff dataset and propose strong baseline called PPDF-Embedding,. A distinctive feature of the D-Rep is its use of replication levels. The dataset annotates each replica into 6 different replication levels. The proposed -DD--mbdddinn first transforms the annotated level into a probability density function (PDF) to smooth the probability. To learn from the PDFs, our PDF-Embedding adopts set of representative vectors instead of traditional representative vector. We hope this work serves as a valuable resource for research on replication in diffusion models and encourages further research efforts in this area.
Disclaimer. The model described herein may yield false positive or negative predictions. Conse- quently, the contents of this paper should not be construed legal advice.
10

---

References
[1] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-Sb: An open large-scall dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-25294 2022.
[2] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. 2021.
[3]
Midjoumey. Midjourney.om,, 2022. Accessed: 2023-10-11
[4] The new bing, 2023. Accessed: October 10, 2023.
[5] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.0612 2022.
[6] Dustin Podell, Zion English. Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint ar Xiiv:2307700955 2023.
[7] Deep-flydd If, 2023. Accessed: October 10, 2023.
[8] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning, pages 16784-16804. PMI.R, 2022.
[9] Katherine Lee, A Feder Cooper, and James Grimmelmann. Talkin ai generation: Copyright and the generative-ai supply chain (the short version). Proceedings of the Symposium on Computer Science and Law, pages 48-63, 2024.
[10] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6048-6058, 2023.
[11] Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan I Lyu. Detecting, explaining, and mitigating memorization in diffussin models. In The Twelfth International Conference on Learning Representations. 2024.
[12] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[13] Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang. Benjamin Hoover, and Duen Homn Chau. DiffusionDB: A large-scall scale prompt gallery dataset for text-to-image generative models. Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 893-911, Toronto, Canada, July 2023. Association for Computational Linguistics.
[14] Wenhao Wang, Yifan Sun, and Yang. benchmark and asymmetrical similarriy learning for practical image copy detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2672-2679, 2023.
[15] Pizzi, Sreya Dutta Roy, Sugosh Nagavara Ravindra, Priya Goyal, and Matthijs Douze. self-supervised descriptor for image copy detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14532-14542, 2022.
[16] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:/807.037745 2018.
[17] Wenhao Wang. Weipu Zhang, Yifan Sun, and Yi Yang. Bag tricks and a strong baseline for image copy detection. arXiv preprini arXiv:21 1tt::2211100004 2021.
[18] Shuhei Yokoo. Contrastive learning with large memory bank and negative embedding subtraction for accurate copy detection. arXiv preprint arXiv:21 2021.
[19] Sergio Manuel Papadakis and Sanjay Addicam. Producing iiinecntation-ivvaiinn embeddings from real-life imagery. arXiv preprint arXiv:2 12.00415, 2021.
[20] Wenhao Wang. Yifan Sun. Zhentao Tan, and Yi Yang. Anypattern: Towards in-context image copy detection. In arXiv preprint 244.:2444..3788. 2024.
[21] Wenhao Wang, Yifan Sun, and Yi Yang. Pattern-exannaable image copy detection. In International Journal of Computer Vision 2024.
[22] Nicolas Carlini, Jamie Hayes, Milad Nast, Matthew Jagielski, Vikash Sehwag. Florian Tramer, Borja Balle, Daphne Ippolito. and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pages 5253-5270 2023.
11

---

[23] Eric Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. arXiv preprint arXiv:2303. 17591, 2023.
[24]
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understanding and mitigating copying in diffusion models. Advances in Neural Information Processing Systems, 2023.
[25] Nikhil Vyas, Sham Kakade, and Boaz Barak. Provable copyright protection for generative models. 2023.
[26] Anonymous. Copyright plug-in market for the texi-to-image copyright protection. In Submitted to The Twelfih International Conference on Learning Reprexentations, 2023, under review.
[27] Yunqing Zhao, Tianyu Pang. Chao Du, Xiao Yang. Ngai-Man Man Cheung, and Min Lin. A recipe for watermarking diffusion models. arXiv preprint arXiv:2303. 10137, 2023.
[28] Yingqian Cui, Jie Ren, Yuping Lin, Han Xu. Pengfei He, Yue Xing. Wenqi Fan, Hui Liu, and Jiliang Tang. Ft-shield: A watermark against unauthorized fine-tuning in text-to-image diffusion models. arXiv preprint 1xXiv2310.02401,222 2023.
[29] Anthony Rhodes, Ram Bhagat, Umur Aybars Ciftci, and like Demir. My art my choice: Adversarial protection against unruly ai. arXiv preprint arXiv:2309.03/ 2023.
[30] Yinggian Cui, Jie Ren, Han Xu. Pengfei He, Hui Liu, Lichao Sun, and Jiliang Tang. Diffusionshield: A watermark for copyright protection against generative diffusion models. arXiv preprint arXiv23066046422 2023.
[31] Ryan Webster, Julien Rabin, Loic Simon, and Frederic Jurie. On the de-duplication of laion-2b. arXiv preprint 23:Xi 2023.
[32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763. PMLR.
2021.
[33] Matthijs Douze, Hervé Jégou, Harsimrat Sandhawalia. Laurent Amsaleg, and Cordelia Schmid. Evaluation of gist descriptors for web-scale image search. In Proceedings of the ACM international conference on image and video retrieval, pages 1-8, 2009.
[34] Matthijs Douze, Giorgos Tolias, Ed Pizzi, Zoč Papakipos, Lowik Chanussot, Filip Radenovic, Tomas Jenicek, Maxim Maximov, Laura Taixé, Ismail Elezi, al. The 2021 image similarity dataset and challenge. arXiv preprint arXiv:2 106. 09672, 2021.
[35] Zoë Papakipos, Giorgos Tolias, Tomas Jenicek, Ed Pizzi, Shuhei Yokoo, Wenhao Wang, Yifan Sun, Weipu Zhang. Yi Yang. Sanjay Addicam, et al. Results and findings of the 2021 image similarity challenge. In NeurlPS 202/ Competitions and Demonstrations Track, pages 1-12. PMLR, 2022.
[36] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, 2019.
[37] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn. Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer. Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
[38] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[39] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.
Icce, 2009.
[40] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347-10357. PMLR, 2021.
[41] Norman Mu, Alexander Kirillow, David Wagner, and Saining Xie. Slip: Self-supervision meets language- image pre-training. In European Conference on Computer Vision pages 544. Springer, 2022.
[42] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-iman pre-training for unified vision- vasion-language understanding and generation. In International Conference on Machine Learning, pages 12888- 12900. PMLR, 2022.
[43] Quan Cui, Boyan Zhou, Yu Guo, Weidong Yin, Hao Wu, Osamu Yoshie, and Yubo Chen. Contrastive vision- lision-language pre-traine training with limited resources. In European Conference on Computer Vision, pages 236-253, Springer, 2022.
12

---

[44] OpenAL. Gpt-4 technical report. 2023.
[45] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pages
1597-1607, PMLR, 2020.
[46] Kaiming He, Xinlei Chen, Saining Xie, Yanghao are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
Dollár, Ross Girshick. Masked autoencoders
[47] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750-15756 15758, 2021.
[48] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
[49] Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po- Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision, 2023.
[50] Mingxing Tan and Quoc Efficientnet: Rethinking model scaling for convolutional neural networks. International conference on machine learning. pages 6105-61
615-61144 14. PMLR, 2019.
[51] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
[52] Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. convnet for the 2020s. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pages 11976-1 1986, 2022.
[53] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
13

---

A More examples of the D-Rep Dataset
We show more example image pairs for each level in Fig. to Fig. 14.
B The Instantiations of PDFs
This section presents examples of PDFs derived from replication levels, focusing on three primary functions: Gaussian, linear, and exponential for calculations, visualization, and analysis. Within the area close to the normalized level p', denoted as 6, the Gaussian function curves downwards making it concave, the linear function is straight with no curvature, and the exponential function curves upwards making it convex. These characteristics indicate the rate at which they deviate from their peak value: the Gaussian function changes slowly in the 6 area, the linear function changes at steady rate, and the exponential function changes rapidly in the area. A fast rate of change suggests the network learns from sharp distribution, while slow rate implies learning from smooth distribution.
Gaussian function. Its general formulation is
g( (4 (- (x - 11)2
(11)
2 2.02
where A >> is the amplitude (the height of the peak), μ [0. 1] is the mean the center, and 0 the standard deviation. To satisfy the requirements of a PDF in Section 4.1, the following must hold:
S.' (A
(x p)²
AA.xx exp
dr =1
2 2.02
(x 80-11 -12
exexp
2
(12)
2. o²
(x - 11)2
(p' u4)2
A A exp
< A exp
2 02
02
From Eqn. 12, we have:
p/
(13)
In practice, with x E (0,0.2,0.400.6.0.. 1) being discrete, the equations become:
Σ
A. (Aeexp/
(-000--2-00 2.02
(I -')-) (-(--p))) 1,
x (0.00.2.0.4.00..........
(14)
(I- (x-2 p!)2
A exp
0.
2.02
Given specific normalized level p' and varying A, g(x o) values are computed for different using numerical approaches. The resulting distributions are visualized in Fig. 15 (top).
Finally we prove that 9(x A, o) is concave for in the interval [μ -i o. + o). This means that in the interested region 5 (near the normalized level p)), its rate of change is slow and increases as x diverges from μ.
Proof: Given the function:
(- (x (a- 11)
g( A, Aexp
(15)
20
we find the first derivative of g with respect to ::
(- (x (x- 11)2
)
g'(x) = dr
A Acxp
(16)
202
Using the chain rule, we have:
g'(x) = Aexp ( (x (u)2
d
(x - 4)
(17)
20²
dr
20
14

---

This gives:
(x (
(x-
g((x) = - exx Next, to find the second derivative, differentiate g'(x) with respect to I:
(18)
20²
o²
d
- Acxp - (- (-6r-1) 11)
(o
- 11)
(a)x) dx
x
(19)
20²
0²
Using product rule and simplifying, the result would be:
(- (2 [-(--)) x
11)
( -
g"(x) exxp
202
(20)
20²
0¹
0²
When (²(x) < O, we have:
o,
(21)
That proves g(x A, j4, o) is concave in the 6 area, and thus its rate of change is slow and increases as x diverges from μ..
Linear function. Its general formulation is
g(x p. A 3- |r 14/,
(22)
where A denotes the maximum value of the function, μ [0, 1] is the point where the function symmetric, and 3 >> 0 determines the function 's slope. To satisfy the requirements PDF in Section 4.1, the following must hold:
(A B l - pl) dr
(23)
A 3 I μ| 2
1r A
A 3 lp p
From Eqn. 23, we have:
//-
(24)
In practice with x E
0.8, }} being discrete, the equations become:
Σ
(A- B - --11) 1,
= =E(0.02.0010660008.11
(25)
1 - p 20. A 3
Given specific normalized level p' and varying A, g(x A, (). 33 values are computed for different I. The resulting distributions are visualized in Fig. (middle).
Finally, we prove that g(x (4,3)) has no curvature, and thus its rate of change is consistent regardless of the value of x.
Proof: Given the function:
(26)
we will differentiate this function based on the absolute value, which will result in two cases for the derivatives based on the sign of (x 14).
Case 1: r > In this case, I - 14| = x. n. So, g( A, 3) A - 3 (x - 11).
First derivative gf (x:)
((x) (A 3. (x - w)) = -3.
(27)
dx
Second derivative (x))
g'(x) d (- 0.
(28)
Case 2: < In this case, --- ul p I. So, g(x A, A 3
8 -4 z).
15

---

First derivative g' (z)
d
g((x) dx
(A B. (p--ap x)) (u-x)) B.
(29)
Second derivative (x)
d (8) 0.
g²(x)
(30)
dx
When the second derivative is constantly 0, it means the function has no curvature and its rate of change constant at every point.
Exponential function. Its general formulation is
g(x A, p, A) A cxppx-xxxx--1 -044)
(31)
where A > denotes the intensity the function, μ E [0,1] is the point where the function is symmetric, and À > 0 determines the spread width of the function. To satisfy the requirements of a PDF Section 4.1, the following must hold:
L (A A exp(-xx -+1) dx = 1,
(32)
A exp( - nl) N 0,
A A exp( - x- pl) < A X eexp( -- -pp' - -1))
From Eqn. 32, we have:
(33)
In practice with x E {0, 0.4, 0.6, 0.8, 1) being discrete, the equations become:
Σ
(A A exp(-> -p'))) = 1,
#e (0.0022004106660088.8..
(34)
A cxp(- - xz -p'1) 2 0.
Given specific normalized level p' and varying A. g(x I., μ. A) values computed for different I using numerical approaches. The resulting distributions are visualized in Fig. 15 (bottom).
Finally, we prove that g( A. u. X) is convex: its rate of change is rapid in the area and decreases as a moves diverges from μ
Proof: Given the function:
g(x A, A) = A- A exp(->- 121-))))
(35)
we find the first and second derivatives with respect x. This function involves an absolute value, which will create two cases for the derivatives based on the sign of (x μ)). Case 1: x > pt In this
case, |ar p n.. So,
g(x A, A x. exp((x (xr n))
(36)
First derivative g((x):
g'(x) = A x exp(- -x (x - p)),
dr
(37)
g'(z) - X2 exp(- - (x H4)
Second derivative )(xx)
d exp( - (r u)), g"(x) A exp(ppp - X (x 4)) 0.
(x)x A x²
dx
(38)
Case 2: x < p.: In this case,
I 11| / 1- I. So,
g(x A, μ, A) = A X exp(- - (μ z)).
(39)
16

---

Cosine Similarity Heatmap
0.18
0.063
-0.098
-0.13
-0.0045
0.8
-0.085
-0.24
0.063
0.1
0.16
0.046
-0.18
0.4
-0.098
-0.085
0.16
1
0.19
-0.16
0.2
-0.13
-0.24
0.046
0.19
0.036
0.0
0.000
-0.11
-0.18
-0.16
0.036
1
0.2
Figure 7: The cosine similarity heatmap of the learned vectors.
First derivative g'(x)
g'(x) A X d exp(-> -- (1 z)),
dr
(40)
g g'(x) A 12 exp(- -- (μ x)).
Second derivative g" (x)
g"(xx) A X d
exp( --p (μ x)), dæ
(41)
d"(x)= A exp( -- (p - x)) 0.
When the second derivative bigger than means the function convex: its rate of change is rapid in the 8 area and decreases as moves diverge from μt
C The Visualization of What the Network Learns
To gain insights into what the network has learned, we offer two visualization methods. First, we present the cosine similarity heatmap of the learned C |c]: ci..... c& (refer Fig. 7). Second, we show the distribution changes of image pairs throughout the training process. The final epoch's 's distribution can be seen in Fig. 16, while the entire training process is depicted in the attached videos.
From the heatmap, we conclude that: (1) The cosine similarity between different vectors very low. This demonstrates that the learned vectors are linearly independent. (2) Neighboring vectors exhibit relatively high cosine similarity. This consistent with the expectation, as they correspond similar replication levels.
From the observed changes in the distributions, we note that: (1) While the distribution initially starts as uniform distribution or peaks at incorrect level, the network, after training, eventually produces an appropriate and accurate distribution for each image pair. (2) For instance, when supervised by the Gaussian distribution, the network, as expected, produces a final distribution that, though not perfect, closely imitates this.
D Implement GPT-4V Turbo on our D-Rep Test Dataset
This section details implementing GPT-4V Turbo our D-Rep test dataset. GPT-4V Turbo, which has been online since November 6, 2023, is the latest and most powerful large multimodal model developed by OpenAL. Because it cannot be regarded as feature extractor, directly prompt with two images and one instruction:
17

---

Give you one pair of images; please give the similarity of the second one to the first one. Diffusion Models generate the second one, while the first one is the original one. Please only reply with one similarity from 1; no other words are needed. Please understand the images by yourself.
Given these prompts, GPT-4V Turbo returns similarity ranging from 0 to 1. Using the official API, we ask the GPT-4V Turbo determine all similarities between the image pairs in the D-Rep test dataset. Note that the computational cost of employing GPT-4V Turbo in practical applications prohibitively high. Specifically, to compare an image against an image database containing one million images, the API must be called one million times, incurring cost of approximately $7, 800.
E
The Similarities Predicted by Other Models
In Fig. 17, we show the similarities predicted by six selected models (two vision-language models, two current ICD models, and two others). We conclude that: (1) CLIP [32] tends to assign higher similarities, which deteriorates its performance on image pairs with low levels, leading to many false positive predictions; (2) GPT-4V Turbo [44] and DINOv2 [49] can produce both high and low predictions, but its performance does not match ours. (3) The prediction ranges of ResNet-50 [53] are relatively narrow, indicating its inability to distinguish image pairs with varying levels effectively. (4) Current ICD models, including SSCD [15] and BoT [17], consistently produce low predictions. This because they are trained for invariance to image transformations (resulting in high similarities for pirated content produced by transformations) and cannot handle replication generated by diffusion models.
F
The Details of Six Diffusion Models
This section provides details on the evaluation sources for three commercial and three open-source diffusion models.
Midjourney [3] was developed by the Midjourney Al company. We utilized dataset scraped by Succinetly AI under the cc0-1.0 1.0 license. This dataset comprises 249, 734 images generated by real users from public Discord server.
New Bing [4], also known as Bing Image Creator, represents Microsoft's latest text-to-image technique. We utilized the repository under the Unlicense to generate 216, 957 images. These images were produced using randomly selected prompts from DiffusionDB [13].
DALLE-2 [5] is a creation of OpenAI. We downloaded all generated images directly from this website, resulting in a dataset containing 50, 904 images. We have obtained permission from the website's owners to use the images for research purposes.
We downloaded and deployed three open-source diffusion models, including SDXL [6], DeepFloyd IF [7], and GLIDE [8]. These models were set up on local server equipped with A 100 GPUs. Distributing them, we generated 1, 819, 792 images with prompts from DiffusionDB [13].
G More Replication Examples
We provide more replication examples by diffusion models in Fig. 18 to Fig. 23.
H Failure Cases
As shown in Fig. 8, we identify two primary failure cases. The first type of failure occurs when the generated images replicate only common elements without constituting replicated content. For instance, elements like grass (Midjourney). helmets (New Bing), and buildings (DALL-E) 2) appear frequently but do not indicate actual replication of content. The second type of failure arises when two images share high-level semantic similarity despite having no replicated content, An example can be seen in the image pairs where themes, styles, or concepts are similar, such as the presence of iconic structures (SDXL and GLIDE) stylized portraits (DeepFloyd IF), even if the specific
18

---

Midjourney
New Bing
DALL-E
SDXL
DeepFloyd IF
GLIDE
Generated
Matched
Figure 8: The failure cases of our detection method. We show one example for each diffusion model.
content is not replicated. Understanding these failure modes is crucial for improving the accuracy and robustness of our detection methods in the future.
19

---

Figure 9: The example image pairs with level 5.
20

---

OXERR OR
Figure 10: The example image pairs with level 4.
21

---

EARI
Figure 11: The example image pairs with level 3.
22

---

the Lastt L2
Figure 12: The example image pairs with level 2.
23

---

Cp
VITEN BOSTT HEASS PET
Figure 13: The example image pairs with level 1.
24

---

CANND'Y
ATIIN
Figure 14: The example image pairs with level 0.
25

---

@01A, Acp
022
0.22
0.23
3.20
0.0
0.00
0016
1.15
8.0
0.6
2286
06.3 22755
0.53
62.3
1866
68.0 0366
0.35
0.35
0-0AT
1654
1-
Lend
levi
0-7
A-03
A-
0.30
0325
0323
0.22
3300
63125
60123
3100
8-26.5 1-2655-0.007
2- 00011
0300
1- 1-0---0007
2-26.- 0644
0300
2-33.-
2- 1310
Lond
Lead
0.2
A-024
0.23
3.200
5.000
0.00
51199
10379
10300
1100
0.00
0300
1-12.3-0265
0.145
3012
02030
(.A) Arteerg (-2-d
020
0.30
3.10
6.18
3
934
3-122-6440
3--22 1300
GBL 3310
0.24
932
0.22
0.22
0.2
0.21
0.20
0.00
0.20
3.10
0.18
2.36
2016
0.06
036
3-31.2 63257
2.222 6320
1-06.1-
034
2-11.- 11111-1007
1993
06.
034
032
032
0,2
Lond
Figure 15: The distributions converted from replication levels. We use Gaussian, linear, and exponen- tial functions the representative demonstrations.
26

---

24
0.15
Gentrtod
0.17
Probaliity
Probubalty
0.16
0.10
0.14
Levels
Leve
0.18
0.18
Prubuciity
0.16
0.15
0.15
Levels
Epoch 24
Leve 3
Epoch - 24
Leve
0.18
0.18
Generatad
0.17
0.17
Probabiity
0.15
0.15
0.34
0.14
Epoch 24
Love 2
Epoch 24
Leve
Genzated
0.17
0.17
Probabiity
0.
0 0.6
Roal
0.14
0.14
Levels
Leves
Epoch
24
0.18
0.16
Gonortted
0.17
Probabiity
0.16
0.16
Roal
0.15
Levels
Leve
Epoch
Epoch
0.19
0.19
0.18
Robabiing
0.16
0.15
0.15
014
0.14
Figure 16: The learned distributions different image pairs. Please see the attached videos for distribution changes in the whole training process.
27

---

Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.63
0.68
0.85
0.87
GPT-4V
0.40
0.60
0.30
0.90
DINOv2
0.35
0.65
0.69
0.35
ResNet-50
0,54
0.61
0.53
0.58
SSCD
0.04
0.16
0.08
0.10
BoT
0.05
0.36
0.21
0.06
Label
1.00
1.00
1.00
1.00
Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.49
0.57
0.65
0.68
GPT-4V
0.70
0.20
0.00
0.30
DINOv2
0.40
0.60
0.21
0.44
ResNet-50
0.67
0.56
0.65
0.56
SSCD
0.11
0.13
0,00
0.12
BoT
0.24
0.36
0.11
0.25
Label
0.80
0.80
0.80
0.80
Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.70
0.71
0.82
0.93
GPT-4V
0.70
0.20
0.30
0.80
DINOv2
0.61
0.59
0,40
0.76
Re
0.60
0.60
0.57
0.62
SSCD
0.08
0.02
0.08
0.31
BoT
0.25
0.03
0.22
0.35
Label
0.60
0.60
0.60
0.60
Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.57
0.57
0.62
0.77
GPT-4V
0.20
0.30
0.10
0.30
DINOv2
0.28
0.58
0.23
0.55
ResNet-50
0.67
0.60
0.63
0.66
SSCD
0.01
0.14
0.15
0.10
BoT
0.21
0.21
0.25
0.16
Label
0.40
0.40
0.40
0.40
Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.39
0.5S
0.56
0.69
GPT-4V
0.00
0.00
0.00
0.00
DINOv2
0.07
0.17
0.11
0.11
Re
0.59
0.59
0.59
0,56
SSCD
0,06
0.02
0.05
0.12
BoT
0.15
0.01
0.05
0.05
Label
0.20
0.20
0.20
0.20
Method
Sim.
Sim.
Sim.
Sim.
CLIP
0.38
0.56
0.57
0.78
GPT-4V
0,00
0.00
0.00
0.10
DINOv2
0.06
0.02
0.09
0.38
Re
0.61
0.62
0.53
0.63
SSCD
0.05
0.05
0.04
0.23
BoT
0.12
0.08
0.03
0.43
Label
0.00
0.00
0.00
0.00
Figure 17: The similarities (or normalized levels) predicted by existing models.
28

---

Figure 18: The replication examples generated by Midjourney [3].
29

---

TRARS
Figure 19: The replication examples generated by New Bing [4].
30

---

A
Wite fs tre
Figure 20: The replication examples generated by DALLE-2 [5].
31

---

Figure 21: The replication examples generated by SDXL [6].
32

---

Figure 22: The replication examples generated by DeepFloyd IF [7].
33

---

Figure 23: The replication examples generated by GLIDE [8].
34

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We follow this common practice for writing paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the Disclaimer at the end of our main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
35

---

Justification:
This is not theory paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: Yes]
Justification: We provide the detail of our deigned model the main paper and will release the code upon the acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
36

---

Answer:
[Yes]
Justification:
The data and code are publicly available at https://icdiff. wit.au.b.ggtt......... tttthuiiio.....
Guidelines:
The answer NA means that paper does nfot include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/ Sttpc//guidss//ooeeee
aasssnnoooo onooonoooooooo olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips.co. wttpccccppppccccctt.ce..cc.cclct wwtpc//gggge/eeeeeeeeeeeeeeeeeee
/tteggguudde//oeuuubiiiiiiie //t////ooooS/ssseeeeeee cesssummissionoi ieriiissssnoooooio onooiicc))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The experimental settings and details are provided in the main paper well as the Appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification: All the experimental results are significant and stable, and error bars are not reported because would too computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by eπror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
37

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include the experiments compute resources in the section of Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: btttps://neurppae htc///ublc//Etiiiice csGuidel ines?
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see the end of Conclusion and the Disclaimer.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
38

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite all the assets used in our paper, and all the corresponding licenses are mentioned and respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
39

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The data and code are publicly available at https: .ith.g..tth...... io/.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: We only involve crowdsourcing to curate our dataset. The salaries of workers in the data curation process are paid beyond the minimum wage. Our research has nothing do with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do not involve research with human subjects and there is no potential risk.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
40


Reviews:
[{"review_id": "Mc2C92XoMc", "reviewer": "Reviewer_2Ky2", "summary": "This paper proposed a new image copy detection model for diffusion models. A dataset of replication labels (0 to 5) is collected using human labelers and is used to train a model for replication grading. The proposed model estimates a pdf of replication labels and minimizes the error with a pdf representation of class labels. The problem is timely, and the proposed solution is novel.", "questions": "* I don't see why the authors did not choose continuous labels instead of the 6-categories approach. Instead of using argmax in eq(10), a \nweighted average score that is normalized between 0-1 will be more appropriate for practical applications. \n\n* Some details of the labeling processing are missing: how many images did each labeler label? What is the variance between the labelers for a specific image?\n\n* Evaluation can be more robust using a separate dataset with a different prompt/image generation procedures.", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "A new model and framework for image copy detection (ICD) is proposed in the paper. The paper is well-written, and the results indicate a significant step forward compared to the mentioned literature.", "weaknesses": "* The use of continuous labels 0-1 will be more useful in practice. \n* Some details of the labeling procedure are missing: how many labels per image, variance for each labeler, and across labelers. \n* The same dataset D-Rep is used for evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We sincerely thank you for your positive feedback and helpful suggestions. We address your questions below.*\n\n**Q1. The use of continuous labels 0-1 will be more useful in practice.**\n\nA1. Thanks. According to your suggestion, we implement this idea and find that it improves the original performance while allowing for continuous predictions. For instance, the PCCs for three variants of our PDF-Embedding change from ($53.7\\\\%$, $54.0\\\\%$, and $56.3\\\\%$) to ($55.1\\\\%$, $55.9\\\\%$, and $57.7\\\\%$), respectively. We will also include this more useful implementation when we release the code.\n\n**Q2. Details of the labeling procedure are missing.**\n\nA2. Thanks. Currently, we have 10 professional labelers and 40,000 image pairs. Initially, we assign 4,000 image pairs to each labeler. If labelers are confident in their judgment of an image pair, they will directly assign a label. Otherwise, they will place the image pair in an undecided pool. On average, each labeler has about 600 undecided pairs. Finally, for each undecided pair, we vote to reach a final decision. For example, if the votes for an undecided pair are 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, the final label assigned is 3. As a result, we do not calculate the variance for each image, and we believe this process ensures high-quality labeling.\n\n**Q3. Evaluation with a separate dataset.**\n\nA3. We appreciate this constructive suggestion and provide the required experimental results in the table of the attached PDF, along with the associated analysis in the “Common Concerns” section.\n***These experimental results validate the generalizability of our model across different diffusion models.***", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YHVf45Tebq", "reviewer": "Reviewer_dJ32", "summary": "This paper introduces a novel method named ICDiff for detecting whether images generated by diffusion models replicate the training set. The authors have constructed a new dataset called D-Rep and proposed a new embedding method called PDF-Embedding. This approach transforms the replication level of image pairs into a probability density function, using this as a supervisory signal to train the model. The experimental results demonstrate that ICDiff outperforms many existing image copy detection method.", "questions": "1.\tIntuitively, we would use a single set of vectors to characterize the similarity between images, with larger dot product values indicating higher similarity and smaller ones indicating the opposite. This paper, however, uses six sets of vectors to measure six levels of similarity, which seems counterintuitive. Specifically, when two images are almost identical, the set of vectors indicating their similarity as zero needs to be quite different; when two images are completely unrelated, the set of vectors indicating their similarity as zero needs to be as consistent as possible. I am curious about the rationale behind this design.\n2.\tAlthough ICDiff performs well on the D-Rep dataset, its generalizability to other datasets or different types of diffusion models has not been verified. In particular, I am concerned whether other methods compared with ICDiff (such as SSCD) have been trained on the D-Rep training set? If so, how was the training conducted; if not, is this comparison on the D-Rep test set somewhat unfair?\n3.\tRegarding the design of the Relative Deviation (RD). The authors point out in Appendix B that when $s^l=3,s^p=0$, the penalty should be greater than when $s^l=5,s^p=2$; my question is, when $s^l=5,s^p=0$, should the penalty be greater than when $s^l=3,s^p=0$ (although both can't be more wrong, the former seems to be more egregiously wrong)?\n4.\tHow do the authors utilize the trained ICDiff to assess the replication ratio of diffusion models? The paper seems to only mention conclusions (such as 10.91% or 20.21%) without explaining how these figures are derived. Specifically, ICDiff provides six replication levels for each set of images. When assessing the replication ratio of diffusion models, what criteria or which level does the author use as a threshold?", "limitations": "No limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tInnovation: ICDiff is the first image copy detection method specifically aimed at replicas generated by diffusion models, filling a gap in current research.\n2.\tDataset Construction: The creation of the D-Rep dataset provides a valuable resource for the study of image copy detection, with its replication level annotations offering clear guidance for model training and evaluation.\n3.\tMethodology: The PDF-Embedding method, which utilizes a probability density function as the supervisory signal, is both innovative and effective.", "weaknesses": "See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We sincerely thank you for your positive feedback and helpful suggestions. We address your questions below.*\n\n**Q1. The rationale behind using six vectors.**\n\nA1. We appreciate this insightful question. As you say, when we understand our PDF-Embedding approach ***separately***, it seems counterintuitive: Typically, we use **larger** dot product values to indicate higher similarity. In contrast, in the context of PDF-Embedding, when two images are more similar, the dot product of a certain set of vectors, which indicates their similarity as zero, needs to be **smaller**.\n\nHowever, as the name of our method, Probability-Density-Function, suggests, under ideal conditions, our method is ***continuous*** and should not be focused on local regions. From the perspective of continuity, the rationale or intuition is that the probabilities of neighboring replication levels should be continuous and smooth. For instance, if an image-replica pair is annotated as level-3 replication, the probabilities for level-2 and level-4 replications should not be significantly low either. After training, the largest-scored entry indicates the predicted replication level. This intuition is experimentally verified by comparing our method against two standard supervision signals, namely “One-hot Label” and “Label Smoothing,” as shown in Table 2.\n\nFurthermore, even if we only focus locally, learning such features by optimizing the neural network will **not** cause contradictions. The neural network is capable of capturing the underlying distribution and continuity, ensuring that the learned embeddings reflect the smooth transition between different levels of similarity.\n\n **Q2. Generalizability to other datasets or diffusion models.**\n\nA2. We provide the required experimental results in the table of the attached PDF, along with the associated analysis in the “Common Concerns” section. ***These experimental results validate the generalizability of our model across different diffusion models.***\n\n**Q3. Table 1: Somewhat unfair comparison with other models.**\n\nA3. We apologize for causing this confusion. The purpose of Table 1 is to demonstrate that ***all existing models fail to solve the proposed ICDiff task, highlighting the necessity for a specialized ICDiff model*** rather than to compare our model against them. For details, please refer to the “Common Concerns” section.\n\n**Q4. The design of Relative Deviation (RD): it does not reflect more egregiously wrong in some cases.**\n\nA4. We appreciate your deep understanding of our designed Relative Deviation metric. Indeed, the Absolute Deviation (Eq. 12) indicates that when  $s^l = 5$  and  $s^p = 0$, the penalty is greater than when $s^l = 3$ and  $s^p = 0$. Although we notice it is difficult to design an evaluation metric suitable for all cases, both Relative Deviation and Absolute Deviation fulfill the primary purpose of proposing the second evaluation metric. Specifically, as mentioned in Lines 132 to 134:\n\n*A limitation of the PCC is its insensitivity to global shifts. If all the predictions differ from their corresponding ground truth with the same shift, the PCC does not reflect such a shift and remains large. To overcome this limitation, we propose a new metric called the Relative Deviation (RD).*\n\nBoth Relative Deviation and Absolute Deviation can reflect global shifts: ***for the same label, a larger distance from the label results in a higher deviation/penalty score***.\n\nFurthermore, if we maintain when  $s^l = 5$  and  $s^p = 0$, the penalty is greater than when $s^l = 3$ and  $s^p = 0$, ***we cannot normalize the penalty to the range  $0 \\sim 1 $ for each pair, which may lead to overly optimistic results.*** This is because: assuming the maximum penalty is $1$, we have\n\\begin{equation}\n1=penalty\\left( s^{l}=5,s^{p}=0 \\right) >penalty\\left( s^{l}=3,s^{p}=0 \\right).\n\\end{equation}\n\nIn conclusion, while we acknowledge that our Relative Deviation may not reflect more egregious wrong in some cases, it empirically aligns better with human intuition. Additionally, we will also seek more perfect evaluation metrics.\n\n\n**Q5. How to assess the replication ratio of diffusion models.**\n\nA5. Thank you for your reminder, and we apologize for missing this important detail. When assessing the replication ratio of diffusion models, we consider image pairs rated at Level 4 and Level 5 to be replications. We will add this to the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yE227hhAU6", "reviewer": "Reviewer_2jDa", "summary": "This paper constructs a Diffusion-Replication dataset aiming to solve the image copy detection problem for diffusion models. This paper proposed a strong baseline named PDF-Embedding which transforms the replication level into a probability density function as the supervision signal. Extensive experimental results and analysis demonstrate the efficiency of the proposed method.", "questions": "1. Authors use different values of A for eq3-5, but there is no explanation on both the selection rule and influence of A.\n2. The details of the data set, such as the label distribution of the dataset, are missing.", "limitations": "Yes, the authors addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The topic is timely and interesting. The presentation is clear and easy to follow. \n2. This paper creates a valuable dataset, which is important to identify the replication caused by diffusion models.\n3. This paper gives a reasonable theoretical explanation for the proposed method. The analysis of the proposed PDFs is convincing on three primary functions: Gaussian, linear, and exponential.\n4. The experiments are intensive and reasonable results are achieved.", "weaknesses": "1. As shown in Figure 5 and Figure 15, authors use different values of A for each function, while there is no explanation on both the selection rule and influence of A.\n2. As shown in Table 2, the training time, inference time and matching time are not better than other methods.\n3. This paper creates a valuable dataset, while the details of the data set, such as the label distribution of the dataset, are missing, which is important in practical applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We sincerely thank you for your positive feedback and helpful suggestions. We address your questions below.*\n\n**Q1. The selection rule and influence of $A$.**\n\nA1. We thank you for this insightful question. We provide the selection rule and influence of $A$ here.\n\n**Selection rule.**\n\n***We select $A$ to maintain $g \\left( x \\right )$ as a validation PDF.*** Because the random variables are discrete in practice, $A$ cannot be randomly selected, and its value must lie within a certain range. The below example demonstrates how to calculate the range of $A$:\n\nFor the *Gaussian* function, according to Eq. 16: \n\\begin{equation}\n\\begin{gathered}\\sum_{x \\in\\{0,0.2,0.4,0.6,0.8,1\\}}\\left(A \\cdot \\exp \\left(-\\frac{\\left(x-p^l\\right)^2}{2 \\cdot \\sigma^2}\\right)\\right)=1, \\\\\\\\ A \\cdot \\exp \\left(-\\frac{\\left(x-p^l\\right)^2}{2 \\cdot \\sigma^2}\\right) \\geq 0, \\end{gathered}\n\\end{equation}\nwe have: \n\\begin{equation}\n6 \\cdot A \\geq \\sum_{x\\in \\{ 0,0.2,0.4,0.6,0.8,1\\}} \\left( A\\cdot \\exp \\left( -\\frac{\\left( x-p^{l} \\right)^{2}}{2\\cdot \\sigma^{2}} \\right) \\right) =1.\n\\end{equation}\nTherefore, we have $A \\geq \\frac{1}{6}$. As shown in Fig. 15, experimentally, we start with $A=0.167$. Similarly, we can calculate the range of $A$ for the *Linear* and *Exponential* functions.\n\n**Influence.**\n\nAs shown in Fig. 15, for the *Gaussian* and *Linear* functions, a larger $A$ implies steeper supervision, while for the *Exponential* function, a larger $A$ implies smoother supervision. According to our intuition that “the probability of neighboring replication levels should be continuous and smooth,” the parameter $A$ controls the expected smoothness of the learned distribution.\n\n***We thank you again for this insightful question, which helps make our method more theoretically sound. We will incorporate these points into our revised paper.***\n\n\n**Q2. The efficiency training and testing is not better than others.**\n\nA2. Thanks for this question. Since we use a set of vectors to calculate similarity instead of the original one, it is reasonable to expect little computational overhead. However, in our paper, we argue that ***the computational overhead introduced by our method is negligible***. Specifically: (1) during training, our method is only $5.8\\\\%$ slower than the baseline; (2) during inference, our method is only $2.8\\\\%$ slower than the baseline; (3) the magnitude of matching, which is $10^{-9}$, is negligible compared to the magnitude of inference, which is $10^{-3}$. Furthermore, as shown in Lines 262 to 266, we find that in practice:\n\n*Our PDF-Embedding requires only $2.07 \\times 10^{-3}$  seconds for inference and an additional $8.36 \\times 10^{-2}$ seconds for matching when comparing a generated image against a reference dataset of 12 million images using a standard A100 GPU. This time overhead is negligible compared to the time required for generating (several seconds).*\n\nIn conclusion, ***given the significantly enhanced performance, the introduction of such a minimal computational overhead is worthwhile in practice.***\n\n**Q3. The details of the proposed dataset, such as the label distribution.**\n\nA3. Thanks. We have provided the label distribution of the proposed dataset on the left side of Fig. 3 in the main paper. To make this clearer, we will highlight it in the main text. Additionally, we provide more labeling details in the response to **Q2** of *Reviewer 2Ky2*, which enriches the details of the proposed dataset.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RFCrphqwPT", "reviewer": "Reviewer_vdEZ", "summary": "This paper studies a novel task of Image Copy Detection for Diffusion Models (ICDiff). Different from traditional Image Copy Detection task, ICDiff focuses on detect and evaluate the degree of image copy for images generated by text-2-image (T2I) generative diffusion models. This is an important and meaningful task for the community to study, with the emergence and influence of T2I diffusion models on downstream tasks, such as image editing, icon generation, or style transfer. Effectively and precisely detecting the generated images that might \"copy\" content/style/character of the artifically created images in its training data can help the end users avoid violating issues such as Image Copyright.\n\nTo study such a problem, the authors collected and proposed a new dataset called D-Rep. To build such a set, the authors first try to find text prompts from DiffusionDB that are highly similar to image titles from LAION-Aesthetics V2, using Sentence-Bert score. Subsequently, they generate 40k images using these collected prompts using Stable Diffusion 1.5 to simulate possible image copies from LAION. To obtain the degree of image copy annotation, the authors employ human annotators to score each generated-image-&-LAION-image paris from 1 to 6.\n\nFurthermore, the authors propose and train a probablity density based embedding model (PDF-Embedding) to score the degreee of image copy for input images. 90% of the collected D-Rep dataset is used as training set for the proposed method.\n\nIn the experiments, the proposed PDF-Embedding more effectively and efficiently detect image coplies in the D-Rep test set, comparing to the compared methods. However, the quantative experiments are limited to the proposed dataset. In addition, the comparison is somewhat unfair because many of the compared methods (such as GPT-4V, CLIP, DINOv2) are not trained or fine-tuned on the training split of D-Rep, but the proposed method is.", "questions": "Q1: Do the authors study the quality of text matching using Sentence-Bert? Pure linguistic matching could be quite ambigious and subseqeuntly leads to incorrect matching. For instance, \"containers\" can be matched to both the huge containers for shipment at the port and the plastic containers (such as lunch box) in a kitchen context. In addition, do the authors study different scores, such as CLIP text encoder score instead of Sentence-Bert score? As observed in [1], for visual content related text, CLIP encoder seems better than Sentence-Bert.\n\nQ2: How do you define Image Copy for Diffusion Models? What kind of \"Copy\" should be detected? This is important for the studied task. For instance, when a user prompts a T2I diffusion model \"A portrait of the Mona Lisa.\", of course, a highly similar image that copies the famous Mona Lisa potrait will be generated. But, is this an image copy? This kind of detection might not helpful for downstream tasks. The task should be defined better and thereby the method can be designed better. From the visualization in Fig. 3, I found the high-score copies are mainly \"object copy\". These copies might occur frequently if the prompt tasks the diffusion model to generate such an image. By conditioning and limiting the prompt text, this can be solved directly. So why is it useful? T In my opinion, a worth-to-detect image copy would be when prompting \"Generate a yellow cartoon-style mouse\" and a Pikachu-like image is automatically generated without explicitly mentioning it in the prompt.\n\nQ3: How well the proposed method will generalize to images generated by other diffusion models, such as SDXL or DALL-E-3, quantatively?\n\n\n[1] Chen, Z., Chen, G. H., Diao, S., Wan, X., & Wang, B. (2023). On the Difference of BERT-style and CLIP-style Text Encoders. arXiv preprint arXiv:2306.03678.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- [Clear writing]: The writing is clear. The sturcture of the paper is easy to follow.\n- [Novel and Important Task]: The studied and proposed task Image Copy Detection for Diffusion Models (ICDiff) is novel, practical, and meaningful. It is important for the community to address the Image Copy issues caused by the training data of T2I diffusion models. This will significantly help downstream tasks avoid copyright issue.\n- [Good baseline]: The proposed PDF-embedding is an efficient and good baseline to address the proposed task.", "weaknesses": "- [Incompleteness]: This paper is not complete in itself. The authors largely use \"refer to Section XXX in the Appendix\" throughout the paper. At Sec. 5.1 (Line#185), the entire section for Training Details contains only 1 sentence \"Please refer to Section E in the Appendix\". I understand the space is limited in the main manuscript and the authors want to use the space for more important content. However, a paper should be complete in itself. After carefully reading the paper, I cannot know: i) how the model is trained, what is the architecture, what are the configuration; ii) how the other methods are compared, do the authors train them, how are they trained or adapted to the current task? These details are important for the main paper for the readers to understand the method and the experiments. \n\n- [Limited experiments]: The quantitaitve experiments are only conducted on one dataset (4k images in the test set) in Tab. 1. In addition, this dataset is generated by only one specific T2I model Stable Diffusion 1.5. Different generative models might have different image copy patterns. The current experiments can hardly validate the effectiveness and advantages of the proposed method over other methods. More comprehensive experiments should be conducted. When switching prompts (data) and the generative models, maybe zero-shot methods such as CLIP, M-LLMs (GPT4-V), or DINOv2 are more generalize. Without large scale experiments, it is hard to draw a conclusion.\n\n- [Unfair comparison]: In Tab. 1, the compared Vision-Language Models (CLIP, GPT-4) and Self-supervised Learning Models (e.g., DINOv2) are not trained or fine-tuned on the evaluation dataset's training split. However, the proposed method is trained on the training split. This makes the comparison unfair. To ensure a fair comparison, either the compared methods should be trained or fine-tuned on the same training split; or, evaluate all methods on novel datasets in a zero-shot manner. With the current unfair comparison, the advantage and effectiveness of the proposed method is not clear.\n\n- [Duplicate content]: Sec. 5.2 and 5.3 are highly overlapped.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*We sincerely appreciate your positive feedback and helpful suggestions. We hope our paper will meet your approval once we address the following concerns.*\n\n**Q1. The large use of \"refer to Section XXX in the Appendix\".**\n\nA1. Due to limited space, we indeed have placed the ***expected implementation details*** in the Appendix. Since the camera-ready version will have ***one more page***, we will move these details into the main paper. We also summarize them as below:\n- *How the model is trained, what is the architecture, what are the configurations?*\n```\nWe implement our PDF-Embedding using PyTorch and distribute its training over 8 A100 GPUs. The ViT-B/16 serves as the backbone and is pre-trained on the ImageNet dataset using DeiT. We resize images to a resolution of 224 × 224 pixels before training. A batch size of 512 is used, and the total number of training epochs is 25 with a cosine-decreasing learning rate. \n```\n- *How the other methods are compared, do the authors train them, how are they trained or adapted to the current task?*\n```\nIn Table 1, we apply the methods in a zero-shot manner. Specifically, we use these models as feature extractors and calculate the cosine similarity between pairs of image features, except for GPT-4V Turbo. We ask GPT-4V Turbo to reply with one similarity directly.\n\nFor the five different methods compared in Table 2, we ensure a fair comparison with the proposed method by using the same training schedule, network architecture, and configuration as described above.\n```\n\n**Q2. Generalizability to other datasets or diffusion models.**\n\nA2. We provide the required experimental results in the table of the attached PDF, along with the associated analysis in the “Common Concerns” section. ***These experimental results validate the generalizability of our model across different diffusion models.***\n\n**Q3. Table 1: Unfair comparison with other models.**\n\nA3. We apologize for causing this confusion. The purpose of Table 1 is to demonstrate that ***all existing models fail to solve the proposed ICDiff task, highlighting the necessity for a specialized ICDiff model*** rather than to compare our model against them. For details, please refer to the “Common Concerns” section.\n\n**Q4. The overlap between Sec 5.2 and Sec 5.3.**\n\nA4. We will delete the first part, which contains an inappropriate/unfair comparison, in Section 5.3.\n\n**Q5. The quality of text matching using Sentence-Bert.**\n\nA5. ***We are aware that relying solely on linguistic matching can lead to incorrect matching; however, this does not affect the quality of our D-Rep dataset.*** This is because the text matching is used as a ***pre-filtering*** step, and we engage ***professional labelers*** after identifying candidate pairs. Specifically, as shown in Line 113 ~ Line 121, these labelers assess the pairs from a visual perspective; for example, they distinguish between large shipping containers and small plastic kitchen containers, assigning a replication level of $0$. As Fig. 3 illustrates, only $15.92\\\\%$ of the dataset consists of image pairs at Level $0$, *indicating the number of mismatched pairs is not large*. ***In conclusion, this approach ensures that mismatching pairs also provide supervision signals (as negative samples) to train models instead of compromising dataset quality.***\n\n**Q6. The definition of copy or replication.**\n\nA6. The definition of copy/replication in this paper follows [1]:\n\n***“We say that a generated image has replicated content if it contains an object (either in the foreground or background) that appears identically in a training image, neglecting minor variations in appearance that could result from data augmentation.”***\n\nAccording to [1], this definition “focuses on object-level similarity because it is likely to be the subject of intellectual property disputes”. After clarifying this, we provide a point-to-point reply to your questions.\n- *The generated Mona Lisa is not a copy, and thus this kind of detection is not helpful for downstream tasks.*\n\nWe respectfully disagree with that. First, it is important to note that ***this paper aims to detect replication itself, providing a basis for subsequent copyright checks rather than directly detecting copyright infringement***. Furthermore, ***such cases also cause plausible infringement problems; therefore, detecting them assists in copyright protection.*** Specifically, although the Mona Lisa is in the public domain and not subject to copyright, diffusion models are equally capable of generating other famous works, such as “*The Persistence of Memory*”, which remains under copyright by the Dalí Foundation. This is illustrated in the figure of the attached PDF. Utilizing these generated images for profit would constitute a copyright infringement against the Foundation. ***Therefore, it is reasonable to regard such cases as copies.***\n- *Limiting the prompt solves the copy directly.*\n\nWe respectfully disagree with that. Because: (1) Many open-source diffusion models, like Stable Diffusion, do not have such limits. Therefore, ***users can easily generate copies of copyrighted images.*** (2) ***Although commercial models limit the prompt text, as shown by [2], this approach does not completely prevent object-level copying***. For instance, Fig. 1 in [2] shows copyrighted Superman logo can be generated by ChatGPT without mentioning Superman directly.\n- *Detecting the case without explicitly being mentioned in a prompt.*\n\n***This case will also be solved by our method*** because we focus on ***matching visual features*** rather than ***prompting***. For instance, whether the prompt is ‘Generate a yellow, cartoon-style mouse’ or ‘Generate a Pikachu,’ if the generated image resembles Pikachu, our algorithm can match it with a copyrighted Pikachu image.\n\n[1] Diffusion art or digital forgery? investigating data replication in diffusion models. *CVPR, 2023*.\n\n[2] On Copyright Risks of Text-to-Image Diffusion Models. *Arxiv, 2023*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Thanks and Solve Common Concerns**

We sincerely thank the ACs and reviewers for their dedicated efforts in reviewing our paper. We also thank all reviewers for their positive, thoughtful, and helpful feedback, and we will add all these suggestions to the final version of our paper.

We are encouraged that they found our work to be “novel” (`Reviewer 2Ky2`, `Reviewer dJ32`, and `Reviewer vdEZ`), “important” (`Reviewer 2jDa` and `Reviewer vdEZ`), “timely” (`Reviewer 2Ky2` and `Reviewer 2jDa`), “valuable” (`Reviewer dJ32` and `Reviewer 2jDa`), and “interesting” (`Reviewer 2jDa`). We are also glad that `Reviewer 2jDa` recognized our work as “This paper gives a reasonable theoretical explanation for the proposed method” and “The experiments are intensive and reasonable results are achieved.” Finally, we sincerely appreciate that `Reviewer 2Ky2`, `Reviewer 2jDa`, and `Reviewer vdEZ` think our paper is “well-written”/“clear”/“easy to follow”.

We have thoroughly addressed all the concerns raised by the reviewers in the below separate responses. Here, we want to solve two common concerns regarding **generalizability to other datasets or diffusion models** (`Reviewer 2Ky2`, `Reviewer dJ32`, and `Reviewer vdEZ`) and **unfair comparison** (`Reviewer dJ32`, and `Reviewer vdEZ`).

- **Generalizability to other datasets or diffusion models**

Thanks for this constructive suggestion. According to the suggestions of `Reviewer 2Ky2`, `Reviewer dJ32`, and `Reviewer vdEZ`, we provide the **quantitative** experimental results on 6 unknown diffusion models (in addition to the qualitative results in Fig. 6 in the manuscript) to validate our generalizability further. The experimental results in the **table** of **the attached PDF file** show that our model has good generalizability compared to all other methods:

1. Our PDF-Embedding is more generalizable compared to all *zero-shot* solutions, such as CLIP, GPT4-V, and DINOv2.

2. Our PDF-Embedding still surpasses all other plausible methods *trained* on the D-Rep dataset in the generalization setting.

3. Compared against testing on SD1.5 (same domain), for the proposed PDF-Embedding, there is *no* significant performance drop on the generalization setting. 

**The quantitative evaluation protocol in the attached table:** Because the collection process of the images from some diffusion models (see Appendix H) differs from the process used to build the test set of our D-Rep dataset, it is difficult to label 6 levels in a short time and the proposed PCC and RD are not suitable. In the attached table, we consider a quantitative evaluation protocol that measures the average similarity predicted by a model for given $N$ image pairs, which are manually labeled with the highest level. When normalized to a range of 0 to 1, a larger value implies better predictions. This setting is practical because, in the real world, most people’s concerns focus on where replication indeed occurs. Due to time constraints, our team and labelers manually confirm 100 such pairs for each diffusion model. *Note that, to ensure fairness, the pre-filtering of these pairs was based on an internal model which is not related to any of the models compared*.



- **Unfair comparison**

Thanks for this kind reminder. ***We did not intend to use Table 1 for direct comparison***. The purpose of Table 1 was to investigate the characteristics of our new dataset, D-Rep, by benchmarking it against popular methods. The results show that D-Rep is ***very challenging*** and popular vision-language models (1st row), self-supervised learning models (2nd row), supervised pre-trained models (3rd row), and current ICD models (4th row) ***fail to solve the new task***. The reason is clear: these models are not specifically trained for it. Given this context, our method offers a reasonable solution. Furthermore, Table 2 validates the effectiveness of our method by ***fairly comparing it against five different methods trained on our D-Rep training set***. Additionally, the table in the attached PDF provides a relatively fair comparison to these pre-trained popular methods to further validate the superiority of our method by ***testing these pre-trained popular methods and our method directly on the image pairs generated by unseen diffusion models***. We will revise the caption and description in the manuscript to clarify that Table 1 is not intended for comparison.

Please let us know if you have any additional questions or concerns. We are happy to provide clarification.

Authors of Submission #367

Meta-review: The paper proposes a novel dataset: D-Rep, to study the problem of copy detection in images produced by diffusion models. The paper categorizes the copying levels and defines a probability density function on these levels for training a transformer-based copy detection setup. 

The paper received all positive reviews, with reviewers acknowledging the timeliness and importance of the dataset and the task. There were questions on: i) the validity and usefulness of the proposed copy levels and their representations, ii) on the training and implementation details, as well as iii) organization of the paper. Authors provided a strong rebuttal, addressing the concerns. AC agrees with the reviewers on the importance of the dataset and recommends accept. 

Authors should incorporate all the suggestions provided by the reviewers in the revised paper, and should ensure the final paper is as complete as possible as pointed out by reviewer vdEZ.
Decision: Accept (poster)