Origin Identification for Text-Guided Image-to-Image Diffusion Models
Authors: Wenhao Wang
Yifan Sun
Zongxin Yang
Zhentao Tan
Zhengdong Hu
Yi Yang
Venue: ICML 2025 poster (2025)
Keywords: Diffusion Models
Origin Identification
Primary area: applications->computer_vision
TL;DR: We introduce the OriPID dataset and a generalizable method with theoretical guarantees to identify original images from their text-guided diffusion model translations.

Abstract: Text-guided image-to-image diffusion models excel in translating images based on textual prompts, allowing for precise and creative visual modifications. However, such a powerful technique can be misused for *spreading misinformation*, *infringing on copyrights*, and *evading content tracing*. This motivates us to introduce the task of origin **ID**entification for text-guided **I**mage-to-image **D**iffusion models (**ID$\mathbf{^2}$**), aiming to retrieve the original image of a given translated query. A straightforward solution to ID$^2$ involves training a specialized deep embedding model to extract and compare features from both query and reference images. However, due to *visual discrepancy* across generations produced by different diffusion models, this similarity-based approach fails when training on images from one model and testing on those from another, limiting its effectiveness in real-world applications. To solve this challenge of the proposed ID$^2$ task, we contribute the first dataset and a theoretically guaranteed method, both emphasizing generalizability. The curated dataset, **OriPID**, contains abundant **Ori**gins and guided **P**rompts, which can be used to train and test potential **ID**entification models across various diffusion models. In the method section, we first prove the *existence* of a linear transformation that minimizes the distance between the pre-trained Variational Autoencoder embeddings of generated samples and their origins. Subsequently, it is demonstrated that such a simple linear transformation can be *generalized* across different diffusion models. Experimental results show that the proposed method achieves satisfying generalization performance, significantly surpassing similarity-based methods (+31.6% mAP), even those with generalization designs. The project is available at https://id2icml.github.io.

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Wenhao Wang'
Yifan Sun
Zongxin Yang Zhentao Tan
Zhengdong Hu' Yi Yang?
Change this image to President Joe Biden being assassinated.
Remove the watermark, An overhead view of couple walking hand in hand along sandbar.
Translate the image of a modern urban scene with a wrecked car medieval setting.
Misinformation
(b) Copyright infringement
Evading content tracing
Figure 1: The illustration for misusing text-guided image-to-image diffusion models in several scenarios: misinformation, copyright infringement, and evading content tracing. Specifically: (a) An altered image originally showing Donald Trump aost-assassination is edited to depict Joe Biden instead; (b) The removal of a watermark from a copyrighted beach image, followed by modifications, could assist in escaping copyright checks; (c) An image of Norwegian government building after an explosion is altered to bypass restrictions, which limit the spread of disturbing images.
Abstract
guided Prompts, which can be used train and test potential IDentification models across various diffusion models. In the method section, we first prove the existence linear transformation that minimizes the distance between the pre-tree pre-trained Variational Autoencoder embeddings of gener- ated samples and their origins. Subsequently, it is demonstrated that such simple linear transforma- tion be generalized across different diffusion models. Experimental results show that the pro- posed method achieves satisfying generalization performance, significantly surpassing similarity- based methods (+31 ..6 mAP), even those with generalization designs. The project is available at https://id2is gi thub. ic.
Text-guided Image-to-image diffusion models ex- cel in translating images based on textual prompts, allowing for precise and creative visual modifica- tions. However, such a powerful technique can be misused for spreading misinformation, infring- ing on copyrights, and evading content tracing. This motivates
introduce the task of ori- gin IDentification for text-guided Image-to-image Diffusion models (ID?), aiming to retrieve the original image of given translated query. A straightforward solution to ID2 involves training specialized deep embedding model to extract and compare features from both query and refer- ence images. However, due to visual discrepancy across generations produced by different diffusion models, this similarity-basd based approach fails when training on images from one model and testing on those from another, limiting its effectiveness in real-world applications. To solve this challenge of the proposed ID2 task, we contribute the first dataset and a theoretically guaranteed method, both emphasizing generalizability. The curated dataset, OriPID, contains abundant Origins and
1. Introduction
Text-guided image-to-image diffusion models are notable for their ability to transform images based on textual de- scriptions, allowing for detailed and highly customizable modification. While they are increasingly used in creative industries for tasks such as digital art re-creation, customiz- ing visual content, and personalized virtual try-ons, there are growing security concerns associated with their mis- As illustrated in Fig. 1, for instance, they could be misused for misinformation,, copyright infringement, and evading content tracing. To help combat these misuses, this paper introduces the task of origin IDentiiication for text-guided Image-to-imet to-image Diffusion models (ID?), which aims identify the original image of generated query from
Universitty of Technology Sydney *Zhejiang University e Harvard University 'Peking University, Correspondence to: Yi Yang www.euu.n..............
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Stable Diffusion 2
Stable Diffusion
Stable
Origin
OpenDalle
ColorfulXI.
Kandinsky-3
Kolors
XI.
Diffusion
Transform this historic city view into a futuristic metropolts through the archway.
Transform the delicate botanical sketch into vibrant. fantastical plant glowing under a moonlü night.
The demonstration for visual discrepancy between generated images by different diffusion models. The images generated by various models exhibit distinctive visual features such realistic textures, complex architectures, life-like details, vibrant colors, abstract expression, magical ambiance, and photorealistic elements.
Figure 2:
large-scale reference When the origin is identified, subsequent compensations include deploying factual correc- tions for misinformation, enforcing copyright compliance, and keeping the tracing of target content.
while only achieving 30.5% mAP ColorfuIXI..
To address the generalization challenge in the proposed task, our efforts focus primarily on constructing the first ID2 dataset and proposing a theoretically guaranteed method.
straightforward solution for the proposed ID2 task to employ a similarity this approach (1) fine-tunes pre-trained network by mini- mizing the distances between generated images and their ori- gins, and (2) uses the trained network to extract and compare feature vectors from the queries and references. However, this approach is impractical in real-world scenarios. This is because: for most current popular diffusion models, such as Stable Diffusion 2 (Rombach al., 2022), Stable Diffusion XL (Podell al., 2024), OpenDalle (Izquierdo, 2023), Col- orfulXL (Recoilme, 2023), Kandinsky-3 (Arkhipkin al., 2023), Stable Diffusion 3 (Esser et al., 2024), and Kolors (KolorsTeam, 2024), in a training-fres free manner, text-guided image-to-image translation can be easily achieved by using an input image with added noise as the starting point (in- stead of starting from randomly distributed noise). Further, as shown in Fig. 2, there exists a visual discrepancy across images generated by different diffusion models, i.e., differ- ent diffusion models exhibit distinct visual features. An experimental evidence for such discrepancy is that we can train lightweight classification model, such as Swin-S (Liu et al., 2021), to achieve a top-1 accuracy of 95.9 when classifying images generated by these seven diffusion mod- els. The visual discrepancy presents an inherent challenge of our ID2 i.e., the approach mentioned above fails when trained images generated by one diffusion model and tested queries from another. For instance, when trained on images generated by Stable Diffusion 2, this approach achieves a 87.1% % mAP on queries from Stable Diffusion 2,
new dataset emphasizing generalization. To verify the generalizability, we construct the first ID2 dataset, OriPID, which includes abundant Origins with guided Prompts for training and testing potential IDentiiication models. Specif- ically, the training set contains 100, 000 origins. For each origin, we use GPT-4o (OpenAI, 2024) to generate 20 differ- ent prompts, each of which implies plausible translation direction. By inputting these origins and prompts into Sta- ble Diffusion 2, we generate 2, 0000 000 training images. For testing, we randomly select 000 images as origins from reference set containing 000, 000 images, and ask GPT-4o to generate a guided prompt for each origin. Subse- quently, we generate 5, 000 queries using the origins, corre- sponding prompts, and each of the following models: Stable Diffusion 2, Stable Diffusion XL, OpenDalle, ColorfuIXL, Kandinsky-3, Stable Diffusion 3, and Kolors. The design of using different diffusion models to generate training im- ages and queries is particularly practical because, in the real world, where numerous diffussin models are publicly available, we cannot predict which ones might be misused.
retrieval approach. Specifically,
simple, generalizable, and theoretically guaranteed solution. To solve the generalization problem, we first the- oretically prove that, after specific linear transformations, the embeddings of original image and its translation, encoded by the diffusion model's Variational Autoencoder (VAE), will be sufficiently close. This suggests that we can use these linearly transformed query embeddings to match against the reference embeddings. Furthermore, we demon-
2

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
strate that these kinds of feature vectors are generalizable across diffusion models. Specifically, by using trained linear transformation and the encoder of VAE from one dif- fusion model, we can also effectively embed the generated images from another diffusion model, even if their VAEs have different parameters or architectures (see Section 5.3 for more details). The effectiveness means the similar per- formance of origin identification for both diffusion models. Finally, we implement this theory (obtain the expected linear transformation) by gradient descending metric learning loss and experimentally show the effectiveness and general- izability of the proposed solution.
(Brooks et al., 2023), IP-Adapter (Ye et al., 2023), EDICT (Wallace et al., 2023), and Plug-and-Play (Tumanyan et al., 2023), perform this task in other paradigms. For detailed discussion, please refer to Appendix (Section G).
Security Issues with AI-Generated Content. Recently, generative models have gained significant attention due to their impressive capabilities. However, alongside their ad- vancements, several security concerns have been identified. Prior research has explored various dimensions of these se- curity issues. For instance, (Lin et al., 2024) focuses on detecting Al Al-generated multimedia to prevent its associated societal disruption, and (Wang et al., 2024c) explores repli- cation problems in visual diffusion models. Additionally, (Fan et al., 2023) and (Chen et al., 2023) explore the eth- ical implications and technical challenges in ensuring the integrity and trustworthiness of Al Al-generated content. In contrast, while our work also aims to help address the se- curity issues, we specifically focus on a novel perspective: identifying the origin of a given translated image.
In summary, we make the following contributions:
1. This paper proposes a novel task, origin identification for text-guided image-to-image diffusion models (ID*), which aims identify the origin of a generated query. This task tries alleviate an important and timely security concern, Le., the misuse of text- image- to-image diffusion models. To support this task, we build the first ID2 dataset.
Image Copy Detection. The task most similar to our ID² is Image Copy Detection (ICD), which identifies whether a query replicates the content of any reference. Various works focus different aspects: PE-ICD (Wang et al., 2024b) and AnyPattern (Wang et al., 2024a) build benchmarks and propose solutions emphasizing novel patterns in realistic scenarios; Active Image Indexing (Fernandez et al., 2023) explores improving the robustness of ICD; and SSCD (Pizzi et al., 2022) leverages self supervisd contrastive learning for ICD. Unlike ICD, which focuses on manually-desigue transformations, our ID2 aims to find the origin of query translated by the diffusion model with prompt-
2.
We highlight an inherent challenge of ID?, i.e., the existing visual discrepancy prevents similarity- based methods from generalizing to queries from unknown diffusion models. Therefore, we propose a simple but generalizable method by utilizing tranf-rern embeddings encoded by the VAE. Theoretically, we prove the existence and generalizability of the required linear transformation.
3. Extensive experimental results show (1) the challenge of the proposed ID² task: all pre-trained deep em- bedding models, fine-tuned similarity-based methods, and specialized domain generalization methods fail to achieve satisfying performance; and (2) the effec- tiveness of our proposed method: it achieves 88.8%, 81.5%, 87.3%, 89.3%, 85.7%, 85.7%, and 90.3% mAP, respectively, for Stable Diffusion 2. Stable Diffu- sion XL, OpenDalle, ColorfulXL, Kandinsky-3, Stable Diffusion 3, and Kolors.
3. Dataset
To advance research in ID², this section introduces OriPID, the first dataset specifically designed for the proposed task. The source images in OriPID are derived from the DISC21 dataset (Papakipos et al., 2022), which is subset of the real- world multimedia dataset YFCC100M (Thomee et al., 2016). As result, OriPID is diverse and comprehensive, covering a wide range of subjects related to real-world misinformation, copyright infringement, and content tracing evasion. An illustration of the proposed dataset is shown in Fig. 3.
2. Related Works
Text-guided Image-to-image Diffusion Models. Recent diffusion models, including Stable Diffusion 2 (Rombach et al., 2022), Stable Diffusion XL (Podell et al., 2024), OpenDalle (Izquierdo, 2023), ColorfulXL (Recoilme, 2023), Kandinsky-3 (Arkhipkin et al., 2023), Stable Diffusion 3 (Esser et al., 2024), and Kolors (KolorsTeam, 2024), have brought significant improvements in visual generation. This paper considers using these popular models for text-guided image-to- tonee-ttommageeee translation as SDEdit (Meng et al., 2022), which is common and cost-effective in the real world. We also note that certain methods, such as InstructPix2Pix
Training Set. The training set comprises (1) 100,000 000 ori- gins randomly selected from the 1, 000, 000 original im- ages in DISC21, (2) 2, 000, 000 guided prompts (20 for each origin) generated by GPT-4o (for details on how these prompts were generated, see Appendix (Section B)), and (3) 2, 000, 000 images generated by inputting the origins and prompts into Stable Diffusion 2 (Rombach et al., 2022).
Test Set. We design the test set with focus on real- world/practical settings. On one hand, we use seven popular
3

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Origin
Stable Diffusion 2
Training
Stable Diffusion 2
Stable Diffusion
Stable Diffusion 3
OpenDalle
ColorfuIXI
Kandinsky-3
Kolors
XL
Query
Reference
Figure 3: The images in our dataset, which is diverse and comprehensive. Specifically, it encompasses variety of subjects commonly found in real-world scenarios where issues such as misinformation, copyright infringement, and content tracing evasion occur. For instance, our dataset includes images of nature, architecture, animals, planes, art, and indoor. Note that for simplicity, we omit the prompts here. Please refer to Appendix (Section C) for examples of prompts and generations.
diffusion models, namely, Stable Diffusion 2 (Rombach et al., 2022), Stable Diffusion XL (Podell et al., 2024),
VAE embeddings of an origin and its translation such that their distance is close enough. Generalizability means that the linear transformation trained on the images generated by one diffusion model can effectively applied to another different diffusion model. Finally, show how to train the required linear transformation in practice.
OpenDDale (Izquierdo, 2023), ColorfuIXI. (Recoilme, 2023), Kandinsky-3 (Arkhipkin et al., 2023), Stable Diffusion 3 (Esser al., 2024), and Kolors (KolorsTeam, 2024), to gen- erate queries. This setting well simulates real-world scenar- ios where new diffusion models continuously appear, and we do not know which one is being misused. On the other hand, for each diffusion model, we generate 5, 000 queries to match 1, 000, 000 references inherited from DISC21, This setting mimics the real world, where many distractors are not translated by any diffusion models.
4.1. Existence
Theorem 1. Consider a well-trained diffission model Fi with an encoder & from VAE and text-guided mnage-to-image functionability achieved by denoising noised images. There exists linear transformation matrix W, for any original image and its translation I1, such that:
Scalability. Currently, only use Stable Diffusion 2 to generate training images. However, our OriPID can be easily scaled by incorporating more diffusion models for training, which may result in better generalizability. Furthermore, we only use 100, 000 origins and generate 20 prompts for each origin. Researchers can scale up our dataset by using the entire 000, 000 original images and generating more prompts with the script in Appendix (Section B).
El(g) E W.
Note that omit the flattening operation that trans- forms a multi-dimensional matrix, E (g1) or (o), into a one-dimensional vector.
4. Method
Proof. The proof of Theorem 1 is based on the below lem- mas. Please refer to Appendix (Section A) for the proofs of lemmas. We prove the Theorem here.
To solve the proposed ID2, we introduce simple yet ef- fective method, which theoretically guaranteed and em- phasizes generalizability. This section first presents two theorems regarding existence and generalizability, respec- tively. Existence means that can linearly transform the
Lemma 1. Consider the diffusion model as defined in
Theorem 1. Define ag as the key coefficient regulating

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Table I:
The cos (1)) gained by compared Stable Diffusion 2 against different diffusion models. The experiments are repeated for ten times to calculate mean and standard deviation.
cos (i))
SDXL
OpenDalle 0.996532 +
ColorfuIXL
Kandinsky-3
SD3
Kolors
0.995790 +
0.998436 #
0.999788
0.993256 =
0.991808 =
SD2
0.000037
0.000016
0.000015
0.000009
0.000035
0.000042
servation and lemmas. Please refer to Appendix (Section A) for the proofs of lemmas. We prove the Theorem here.
the noise level. Let denote the noise vector introduced during the diffusion process, and let cg(z,, 1, t.) repre- sent the noise estimated by the diffusion model, where: a denotes the parameters of the model, zy represents the state of the system at time t, and с encapsulates the rext-condiiioinng information. Under these conditions, the following identity holds:
Observation 1. Consider two distinct matrices, W and 22, satisfying Eq. I and Eq. 3, respectively. Let v, denote the vector of all singular values W, where i E [1. Specifically, define v, = ( of of...., of), with each of representing an singular value of W. Despite the inequality W1 * W2. as shown in Table 1, it is observed that:
E1 (91) - EE((0) Vi- (E Eo (Z): (2)
Vor
COS (p) = V1 V2
Lemma 2. Consider the equation AX = O, where A is a matrix. If A approximately equals to zero matrix, i.e., A x O, then there exists an approximate full-rank solution to the equation.
+ +1.
(4)
|
Lemma (Singular Value Decomposition). Any ma- trix A be decomposed into the product of three matrices: A = UEV" where U and V are orthogo- nal matrices, E is a diagonal matrix with non-negative singular values of A on the diagonal, and V is the
Because well-trained diffusion model learns robust fea- tures and associations from diverse data, it generalizes well to inference prompts that are semantically similar to the training prompts. Moreover, the inference prompts here are generated by GPT-4o based on its understanding of the images, thus sharing semantic overlap with the train- ing prompts. As result, the estimated noise Eg (Zz, c) closely approximates the true noise e. This means the dif- ference between them is approximately equals to zero, i.e., e- (z,,t, c) = 0. According to Lemma 1, this results in & (g1) - & (o) x O. Denote T1 as the matrix, in which each column is & (gx) - (o) from a training pair. Ac- cording to Lemma 2 and T1 R O, we have T = 0 has an approximate full-rank solution. That means the matrix W satisfying Eq. exists.
conjugate transpose of V.
Lemma matrix A has a left inverse if and only if it has full rank.
Consider Tl in the proof of Theorem 1, and denote T2 as the matrix, in which each column is E (g2) (o) from a training pair. Therefore, we have T, W, = and T2 W2 = 0. To prove Theorem 2, we only need to prove T2 = O. According to Lemma 3, there exists orthogonal matrices, U,, U2, Vi, and V2, with diago- nal matrices, Σ1 and Eg, satisfying W, U, Vi and W₂ = U2522 Vž. According to Observation 1, there exists a > 0 such that Σ1 = Q Σz. Therefore, we have:
L Note: here we do not show that Ei = (o) (in this case, there would be no need of W); instead, we prove that there exists a W that can further minimize the distance be- tween (g1) and (l(). despite the distance already being small.
W1 = U,E Vi aU a U1 (U;W v2) Vi (U,U;) (V.V). (5)
Let U3 U; and V3- = V2 where Ug and V3 are thus orthogonal matrices. Therefore:
4.2. Generalizability
|| W2 |= a || T2 (U,U;) W2 (V2Vi) II
T2U, W.V.
(6)
Theorem 2. Following Theorem 1, consider a differ- ent well-trained diffusion model F2 and its text rext-guided image-to-image functionability achieved by denoising noised images. The matrix W can be generalized such that for any original image and its translation g2, we have:
<a || T2U,W, || V. |= T2U;W211
According to Lemma 2 and 4, there exists a matrix K, such that KW2 = I. That means there exists M, such that U,W, = W²M. This results in:
( || S ||2 Wall
E (92) W = (o) W.
(3)
(7)
= a T W2 < a T W2 |M]
Proof. The proof of Theorem 2 is based on the below ob-
Considering T2 W. = 0. we have T2 W, = 0.
5

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Origin
Table 2: Publicly available models fail on the OriPID.
Method
mAP
Acc
Generation
Swin-B (Liu et 2021)
3.9
2.7
Supervised
ResNet-50 (He et al., 2016)
4.5
3.0
E
Pre-trained
ConvNeXt (Liu et al., 2022)
4.5
3.1
W
E
Models
EfficientNet (Tan & Le. 2019)
4.6
3.3
ViT-B (Dosovitskiy et al., 2021)
6.2
4.6
Negative
Self-
SimSiam (Chen & He, 2021)
1.8
1.0
MoCov3 (He et al., 2020)
1.2
supervised
The implementation of learning theoretical- expected matrix W. Specifically, in practice, we use gradi- ent descent to optimize a metric loss in order to learn W.
Figure 4:
DINOv2 (Oquab et al., 2023)
4.3
2.9
Learning
MAE (He et al., 2022)
11.6
9.2
Models
SimCLR (Chen et al., 2020)
11.3
9.7
CLIP (Radford et al., 2021)
2.9
1.8
Vision-
SLIP (Mu et al., 2022)
5.4
3.7
4.3. Implementation
language
ZeroVL (Cui et al., 2022)
5.6
3.8
Models
BLIP (Li et al., 2022)
8.3
5.9
As illustrated in Fig. 4, we show how to learn the theoretical- expected matrix W in practice. Consider a triplet (g, o, n), where I the generated image, o the origin used to generate g, and n is a negative sample relative to g. We have:
ASL (Wang et al., 2023)
5.2
4.1
Image Copy
CNNCL (Yokoo, 2021)
6.3
5.0
Delection Models
BoT (Wang et al., 2021)
10.5
8.2
SSCD (Pizzi et al., 2022)
14.8
12.5
AnyPattem (Wang et al., 2024a)
29.1
25.7
E 2u E {o), and, 2n = = (n),
(8)
Generation
Matching
Generation
Matching
where & is the encoder of VAE. Therefore, the final loss is defined as:
L Lmtr (z W, W W);
(9)
(a) Supervised Pre trained Models
(c) Vision language Models
where Lmar is metric learning loss function that aims to bring positive data points closer together in the embed- ding space while pushing negative data points further apart. We use CosFace (Wang et al., 2018) here as Lyer for its simplicity and effectiveness. Using gradient descent, we can optimize the loss function C obtain the theoretically expected matrix W.
(b) Solf-supervieed Learning Models
(d) Image Copy Detection Models
Figure 5: Examples of failure cases for each kind of model.
Training details. We distribute the optimization of the theoretically expected matrix W across 8 NVIDIA A100 GPUs using PyTorch. The images are resized to resolution of 256 256 before being embedded by the VAE encoder. The peak learning rate is set to 3.5 x 10-A, and the Adam optimizer is used.
5. Experiments
5.1. Protocols and Details
Evaluation protocols. We adopt two commonly used evalu- ation metries for our ID2 task: i.e., Mean Average Precision (mAP) and Top-1 Accuracy (Acc). mAP evaluates a model's precision various recall levels, while Acc measures the proportion of instances where the model's top prediction exactly matches the original image. Acc is stricter as it only counts when the first guess is correct.
5.2. The Challenge from ID>
This section benchmarks popular public deep embedding models on the OriPID test dataset. As shown in Table 2 and Fig. 5, we extensively experiment on supervised pre trainra models, self-supervised learning models, vision- language models, and image copy detection models. We use these models as feature extractors, matching query fea- tures against references. The mAP and Acc are calculated by averaging the results of 7 diffusion models. Please re- fer to Table 9 in Appendix for the complete results. We observe that: (1) All existing methods fail on the OriPID test dataset, highlighting the importance of constructing specialized training datasets and developing new methods. Specifically, supervised pre-trut pre-trained models overly focus on category-level similarity and thus achieve a maximum mAP
Dataset details. During testing, the editing strengths for Stable Diffusion 2, Stable Diffusion XL, OpenDalle, Col- orfulXL, Kandinsky-3, Stable Diffusson 3, and Kolors, are 0.9, 0.8, 0.7, 0.7, 0.6, 0.8, and 0.7, respectively. The edit- ing strengths used in testing manually set to prevent significant visual differences between the generated images and the original ones. During training, the editing strength for Stable Diffusion is 0.9. The classifier- free guidance (CFG) scale set to 7.5 for all diffusion models, which is a commonly used value in practice.
6

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Table 3: Our method excels in performance while keeping efficiency. 'mAP" and 'Acc' are in percentage; 'Train', 'Extract', and 'Match' are in 'h', '10- s/img'. and '10-10 s/pair', respectively.
Seen t
Unseen f
Efficiency 4
Manual
edit f
Method
mAP
Acc
mAP
Acc
Train
Extract
Match
mAP
Acc
Similarity
Circle loss (Sun al., 2020)
70.4
61.3
53.9
48.5
1.79
2.81
0.80
76.6
74.5
-based
SoftMax (LeCun et al., 1989)
82.7
78.3
55.0
49.4
2.25
2.81
0.80
76.2
73.2
Models
CosFace (Wang et al., 2018)
87.1
83.2
52.2
46.5
2.43
2.81
0.80
73.1
70.1
General-
IBN-Net (Pan ct al., 2018)
88.6
85.1
54.6
49.0
2.03
3.42
2.14
75.4
72.1
izable
TransMatcher (Liao & Shao, 2021)
65.6
60.3
65.3
60.7
1.84
2.30
941
78.3
76.4
Models
QAConv-GS (Liao & Shao, 2022)
78.8
74.9
75.8
72.3
1.47
2.30
464
74.4
71.9
Embeddings VAE
51.0
47.0
46.9
43.0
1.59
4.25
66.6
64.6
Ours
With Linear Transformation
88.8
86.6
86.6
84.5
0.17
1.59
0.53
86.6
85.5
Upper: Train& Test Same Domain
88.8
86.6
92.0
90.4
0.17
1.59
0.53
Stable Diffission 2
Stable Diffission XL
OpenDalle
ColorfuIXI
Kandinsky-3
Stable Diffusion 3
Kolors
Query
Manched
Figure 6: Examples of matchings achieved by our method.
The origins can still be retrieved despite non-trivial alterations.
embeddinee embeddings encoded by VAEs from different diffussin mod- els vary. For instance, the average cosine similarity of VAE embeddings for 100,000 original images between Stable Diffusion and Kandinsky-3 is close to 0. Additionally, the dimension of the VAE embedding for Stable Diffusion 2
Table 4:
VAE differs between seen and unseen models.
Sim.
SDXL
OpDa
CoXL
Kan3
SD3
Kolor
SD2
Cony.
0.169
0.169
0.169
0.002
0.169
Embeed
0.120
0.121
0.120
0.023
0.120
of 6.2%; self--uupe learning models handle only sub- tle changes and thus achieve a maximum mAP of 11.6%; vision-language models return matches with overall seman- tic consistency, achieving a maximum mAP of 8.3%; and image copy detection models are trained with translation patterns different from those of the ID² task, thus achieving a maximum mAp of 29.1%. (2) Any Pattern (Wang et al., 2024a) achieves significantly higher mAP (29.1%) 1%) and accu- racy (25.7%) compared to other methods. This is reasonable because it is designed for pattern generalization. Although the translation patterns generated by diffusion models in our ID² differ from the manually designed ones in AnyPattem, there remains some generalizability.
4, 096, whereas for Stable Diffusion 3, it is 16, 384.
5.4. The Effectiveness of our Method
This section shows the effectiveness of our method in terms of (1) generalizability, (2) efficiency, (3) robustness, and (4) the applicability in the manual-editing scenarios. The experimental results for 'Unseen' are obtained by averaging the results from six different unseen diffusion models.
Our method much more generalizable than others. In Table 3, we compare our method with common similarity- based methods (incorporating domain generalization de- signs), all trained on the OriPID training dataset. The mAp and Acc for 'Unseen' are calculated by averaging the results of 6 unseen diffusion models. Please refer to Table 10 for the complete results. Fig. 6 and Section E in the Appendix present the successful retrieval results and failure cases of our method, respectively. We make three observations: (1) On unseen data, our method demonstrates significant per- formance superiority over common similarity -based models. Specifically, compared against the best one, achieve su- periority of +31.6% mAP and +35.1% Acc. (2) Although domain generalization methods alleviate the generalization problem, they are still not satisfactory compared to ours (with at least a -0.8% mAP and -9.1% Acc). Moreover, those with the best performance suffer from severe effi-
5.3. VAE differs between Seen and Unseen Models
A common misunderstanding is that the generalizability of our method comes from different diffusion models sharing the same or similar VAE. In Table 4, we demonstrate that the VAE encoders used in our method differ between the diffusion models for generating training and testing images: (1) The parameters of VAE encoders different. For in- stance, the cosine similarity of the last convolutional layer weights of the VAE encoder between Stable Diffusson 2 and Stable Diffusion XL is only 0.169. Furthermore, the number of channels in the last convolutional layer differs between Stable Diffusion and Stable Diffusion (2) The
7

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
naP for Sean Acc for Sean
map for Unseen Acc. For Unsezn
mAr for Scen Acc. for Scon
m640 for Unszon Acc. for Unscon
mAP for Seen Acc for Sean
MAr for unsecn Acc. For Unceen
MAr for Sean
map for Unseon
Acc for Seon
Alc for Unseon
90
90
90
90
85
85
85
85
MAP/ACE. 196)
80
80
80
80
MAP/ACC,
MAP/NcC.
MAP/ACC.
75
75-
75
75
70
70
65-
65-
6.5
65-
60
100%
65%
30%
60
0=0
0=2 d=3
60
256
60
Without
Strength of JPEG Compression
Strength of Gaussian Blur
Different Image Sizes
Without/With Watermark
Figure 7: Our method demonstrates certain level of robustness against different types and intensities of attacks.
ciency issues, as detailed in the next section. (3) On the seen data, we achieve comparable performance with others. Specifically, there is 0.2% mAP and 1.5% Acc superiority compared to the best one.
images as queries and search for them within a pool consist- ing of their origins along with 1, 000, 000 distractor images. The experiments in Table 3 (right) show that: (1) our method generalizes effectively to real-world, manually edited im- ages; and (2) it achieves the best performance compared to all competing methods.
Our method outperforms others in terms of efficiency. Efficiency is crucial for the proposed task, as it often in- volves matching a query against a large-scale database in real-world scenarios. Table 3, we compare the efficiency of our method with others regarding (1) training. (2) feature extraction, and (3) matching. We draw three observations: (1) Training: Learning matrix based on VAE embeddings is more efficient compared to training deep models on raw images. Specifically, our method is 8.6 times faster than the nearest competitor.
5.5. Ablation Study
In this section, we ablate the proposed method by (1) using different VAE encoders, (2) supervising the training with different loss functions, (3) exploring the minimum rank of W, (4) comparing against non-linear transformations, and (5) analyzing the influence of editing strengths.
(2) Feature extraction: Compared to other models that use deep networks, such as VIT (Doso- vitskiy et al., 2021). the VAE encoder we use is relatively lightweight, resulting in faster feature extraction. (3) Match- ing: Compared the best domain generalization models, QAConv-GS (Liao & Shao, 2022), which use feature maps for matching, our method still relies on feature vectors. This leads to an 875x x superiority in matching speed.
Our method is insensitive to the choice of VAE encoder. In Table 5, we replace the VAE encoder from Stable Diffu- sion 2 with two different encoders from Open-Sora (Zheng et al., 2024) and Open-Sora- Soan--Srr---laa (PKU- & etc., 2024), It is observed that, despite using significantly different well- trained VAEs, such as ones for videos, the performance drop is minimal (less than 1%). This observation experi- mentally extends the Eq. 1 from E (91) W = ( W to &g (91) - w E2( W, where &g is an encoder from totally different VAE.
Our method is relatively robust against different attacks. In the real world, the quality of an image may deteriorate during transmission. As shown in Fig. 7, we apply varying intensities of JPEG compression, Gaussian blur, image re- sizing, and watermarking to evaluate the robustness of our method. It is observed that the side effects of these attacks are relatively minor. For instance, for the unseen diffusion models, the strongest Gaussian blur (a = 3) reduces the mAP by only 3.7%, while the strongest compression (30%) decreases the mAp by just 0.3%. Note that our models are not trained with these attacks.
In practice, selecting an appropriate supervision for learning W is essential. In Table 6, we replace the used supervision CosFace (Wang et al., 2018) with two weaker supervisions, i.e. SoftMax (LeCun et al., 1989) and Circle loss (Sun et al., 2020). We observe that switching Circle loss results in drop in mAP for seen and unseen categories by 3.9% and 4.1%, respectively. Furthermore, using Soft- Max leads to mAP drops of 12.7% and 24.2% for the two categories, respectively. We infer this is because: while our thcorems guarantee the distance between a translation and its origin, many negative samples serve as distractors during retrieval. Without appropriate hard negative solutions, these distractors compromise the final performance.
Our method is applicable to real-world scenarios with manually edited images. In this section, we evaluate our method on a real-world dataset, SEED-Data-ddit (Ge et al., 2024), which contains 52, 000 image editing samples. These samples were collected from amateur photographers who posted their images along with editing requests. Photoshop experts then fulfilled these requests, providing the edited im- ages as target images. Experimentally, we (1) de-duplicate to get 10, 274 image pairs; and (2) treat the edited (target)
To improve efficiency, the rank of W can be relatively low. Assume the matrix W has shape of n X m, where n is the dimension of the VAE embedding and m is a hyperpa- rameter. We show that W is approximately full-rank in the
8

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Table 5: Ablation for choices of VAE encoders.
Table 6: Ablation for different supervision losses.
Seen 1
Unseen 1
Seen t
Unseen
VAE
Supervision
mAP
Acc
mAP
Acc
mAP
Acc
mAP
Acc
Open-Sora
86.3
83.5
86.5
84.2
SoftMax
76.1
72.6
62.4
59.0
Tpen-Sora-Plan
88.8
86.4
86.1
84.0
Circle loss
84.9
82.0
82.5
80.4
Stable Diffusion 2
88.8
86.6
86.6
84.5
CosFace
88.8
86.6
86.6
84.5
90
90
MAP/Acc. (%)
mAP/AcC. (%)
mAP for Seen
85
85
Acc. for Seen
mAP for Seen
mAP for Unseen
80
Acc. for Seen
80
Acc. Unseen
mAP for Unseen
75
Acc. for Unseen
75
4096
2048
1024
512
256
128
64
1
6
The rank of W
Number of Layers
Figure 8: The performance change w.r.t the rank of W.
Figure The performance change w.r.t number of layers.
Table 7: Comparison against non-linear transformations.
single convolutional layer and multi-head attention layer. The experiments in Table 7 show that: (1) likely due to underfitting, the simple convolutional layer results in a per- formance drop; and (2) although the multi-head attention layer marginally improves performance on seen images, its performance on unseen images falls behind our method, due to overfitting
Seen
Unseen f
Transformation
mAP
Acc
mAP
Acc
Convolution 37.4
33.8
32.5
29.6
Attention
89.0
87.2
80.7
78.2
Linear 88.8
86.6
86.6
84.5
proof of existence, and expect that m I n in the proof of generalization. Therefore, the rank of W is m. Experimen- tally, n 4, 096, and we explore the minimum rank of W from 4, 096 as shown in Fig. 8. It is observed that: (1) From 4, (966 to 512, the performance remains nearly unchanged. This suggests that we can train a relatively low-rank W to improve efficiency in real-world upplications. (2) It is expected to see a performance decrease when reducing the rank from 512 to 64. This is because a matrix with too low rank cannot carry enough information effectively linearly transform the VAE embeddings.
Our method maintains high performance across most editing strengths. Detailed experiments and analyses are provided in Appendix (Section D).
6. Conclusion
This paper explores popular text-guided image-to-image diffusion models from a novel perspective: retrieving the original image of a query translated by these models. The proposed task, ID?, is both important and timely, especially as awareness of security concerns posed by diffusion mod- els grows. To support this task, we introduce the first ID² dataset, OriPID, designed with a focus on addressing gen- eralization challenges. Specifically, the training set is gen- erated by one diffusion model, while the test set is gener- ated by seven different models. Furthermore, simple, generalizable solution with theoretical guarantees: First, we theoretically prove the existence of linear trans- formations that minimize the distance between the VAE embeddings of query and its original image. Then, we demonstrate that the learned linear transformations general- ize across different diffusion models, i.e., the VAE encoder and the learned transformations can effectively embed im- ages generated by new diffusion models.
Most non-linear transformations lead to overfitting. In the theoretical section, we proved the existence and gener- alization of W using concepts from diffiusion models and linear algebra. A natural experimental extension of this is to use an MLP with activation functions to replace the simple linear transformation (W). Although linear algebra theory cannot guarantee these cases, we can still explore them ex- perimentally. Experimentally, we increase the number of layers from 1 to 7, all using ReLU activation and residual connections. As shown in Fig. 9, we observe overfitting in one type of diffusion model. Specifically, on one hand, the performance on seen diffusion models improves. For exam- ple, with 2 layers, the mAP increases 91.4% (+2.6%), and Ace rises to 89.4% (+2.8%). However, the other hand, a significant performance drop is observed on unseen diffusion models: with 2 layers, the mAP decreases from 86.6% to 80.3% (-63%%, and Acc drops from 84.5% to 77.3% (-7.2%). The performance drop becomes even more severe when using more layers. Beyond that, we also try a
propose
A Acknowledgment
We sincerely thank OpenAl for their support through the Researcher Access Program. Without their generous contri- bution, this work would not have been possible.

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Impact Statement
Ge, Y., Zhao, S., Li, C., Ge, Y., and Shan, Y. Seed-data-edit technical report: A hybrid dataset for instructional image editing, arXiv preprint arXiv:2405. arXiv:2405.04007, 2024.
Our findings hold potentials for the responsible use of AI- generated content. Specifically, this research helps mitigate the growing concerns of misinformation, intellectual prop- erty violations, and content tracing evasion. However, the proposed model may produce false predictions. Therefore, the paper should not be interpreted as legal advice.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Mo- mentum contrast for unsupervised visual representation learning. 2020.
References
Arkhipkin, V., Filatov. A., Vasleev, V., Maltseva, A., Azizov, S., Pavlov, I., Agafonova, J., Kuznetsov, A., and Dimitrov, D. Kandinsky 3.0 technical report. arXiv:2312.03511, 2023.
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. Masked autoencoders are scalable vision learners. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Brooks, T., Holynski, A., and Efros, A. A. Instructpix2pix: Learning to follow image editing instructions. In the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Izquierdo, A. Opendallevl..1 2023.
KolorsTeam. Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis. 2024.
Chen, C., Wu, 7.,, Lai, Y., Ou, W., Liao, T., and Zheng, Z. Challenges and remedies to privacy and security in aige: Exploring the potential of privacy computing, blockchain, and beyond. arXiv:2306.0041 2023.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. Backpropaga- tion applied to handwritten zip code recognition. Neural computation. 1989.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-iman image pre-training for unified lision-language understanding and generation. In International Confer- ence Machine Learning. PMLR, 2022.
Chen, X. and He, K. Exploring simple siamese representa- tion learning. In the IEEE/CVF conference on computer vision and pattern recognition, 2021.
Liao, S. and Shao, I.. Transmatcher: Deep image match- ing through transformers for generalizable person re- identification. Advances in Neural Information Process- ing Systems, 34: 14:1999--200) 2021.
Cui, Q., Zhou, B., Guo, Y., Yin, W., Wu, H., Yoshie, O., and Chen, Y. Contrastive vasion-language pre-training with limited resources. In European Conference on Computer Vision, pp. 253. Springer, 2022.
Liao, S. and Shao, L. Graph sampling based deep metric learning for generalizable person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1359-7368, 2022.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Lin, L.,, Gupta, N., Zhang, Y., Ren, H., Liu, C.-H., Ding, F., Wang, X., Li, X., Verdoliva, L., and Hu, S. Detect- ing multimedia generated by large ai models: A survey. arXiv: 2tl1:2402.00044 2024.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
Liu, Z., Lin, Y., Cao. Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. the IEEE/CVF inter- national conference on computer vision, 2021.
Fan, M., Chen, C., Wang, C., and Huang. J. On the trustwor- thiness landscape of state-of-the-an generative models: A comprehensive survey. arXiv:2307. 16680, 2023.
Liu, Z., Mao, H., Wu, C.-Y,, Feichtenhofer, C., Darrell, T., and Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-11986, 2022.
Fernandez, P., Douze, M., Jégou, H., and Furon, T. Active image indexing. In International Conference on Learning Representations (ICLR), 2023.
10

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022.
similarity optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6398-6407, 2020.
Tan, M. and Lc, Q. Efficientnet: Rethinking model scal- ing for convolutional neural networks. In International conference on machine learning, 2019.
Mu, N., Kirillov, A., Wagner. D., and Xie, Slip: Self- supervision meets language-image pre-training. In Eu- ropean Conference Computer Vision, pp. 529-544. Springer, 2022.
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2):64 73, 2016.
OpenAl. Hello gpt-4o, 2024.
Oquab, M., Darcet, T., Moutakanni, T. Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Howes, R., Huang, P.-Y., and et al. Dinov2: Learning robust visual features without supervision, 2023.
Tumanyan, N., Geyer, M., Bagon, S., and Dekel, T. Plug- and-play diffusion features for text-driven image-to- image translation. In the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, 2023.
Wallace, B., Gokul, A., and Naik, N. Edict: Exact diffusion inversion via coupled transformations. In Conference on Computer Vision and Pattern Recognition, 2023.
Pan, X., Luo, P., Shi, J., and Tang, X. Two at once: En- hancing learning and generalization capacities via ibn-net. In Proceedings of the european conference on computer vision (ECCV), pp. 464-479, 2018.
Wang. H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J.,
Li, 7.,, and Liu, W. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5265-5274, 2018
Papakipos, Z., Tolias, G., Jenicek, T., Pizzi, E., Yokoo, S., Wang. W., Sun, Y., Zhang, W., Yang, Y., Addicam, S., et al. Results and findings of the 2021 image similarity challenge. In NeurlPS 202/ Competitions and Demon- sirations Track, pp. 1-12. PMLR, 2022.
Wang. W., Zhang, W., Sun, Y., and Yang. Y. Bag of tricks and strong baseline for image copy detection. arXiv:2111 1002000000004....100000000000100 2021.
Pizzi, E.,, Roy, S. D., Ravindra, S. N., Goyal, P., and Douze, M. A supervipd descriptor for image copy detection. In the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Wang. W., Sun, Y., and Yang, Y. A benchmark and asymmetrical-similart learning for practical image copy detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 2672-2679, 2023.
PKU- Yuan, and etc., A. Open-sora-plane April 2024.
Podell, D., English, Z., Lacey, K., Blattmann. A., Dockhor, T., Müller, J., Penna, J., and Rombach, R. SDXL: Im- proving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
Wang. W. Sun, Y., Tan, Z., and Yang, Y. Anypat- tern: Towards in-context image copy detection. In arXiv:2404.13780 2024a.
Wang, W., Sun, Y., and Yang, Y. Pattern- exxennaabe image copy detection. International Journal of Computer Vision, pp. 1-17, 2024b.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Wang. W., Sun, Y., Yang. Z., Hu, Z.. Tan, Z., and Yang, Y. Replication in visual diffusion models: A survey and outlook. arXiv:2408.008 000001, 2024c.
Ye, H., Zhang. J., Liu, S., Han, X., and Yang, W. Ip-adapter:
Recoilme. Colorfuixl-ighhningg, 2023.
Text compatible image prompt adapter for text-to-image diffusion models. 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Yokoo, S. Contrastive learning with large memory bank and negative embedding subtraction for accurate copy detection. arXiv:2112.04333,, 2021.
Zheng, Z., Peng, X., Yang, T., Shen, C., Li, S., Liu, H., Zhou, Y., Li, T., and You, Y. Open-sora: Democratizing efficient video production for all, March 2024.
Sun, Y., Cheng, C., Zhang, Y., Zhang, C., Zheng, L., Wang, Z., and Wei, Circle loss: A unified perspective of pair

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
A. Proofs of Lemmas
and V E R and diagonal matrix Σ Rmxx with non- negative singular values, such that, A UEV* Therefore,
Lemma 1. Consider the diffusion model as defined in Theorem Define di as the key coefficient regulating the noise level. Let denote the noise vector introduced during the diffusion process, and let Eg Eg(2t, 1. repre- sent the noise estimated by the diffusion model, where: 0 denotes the parameters of the model. Zy represents the state of the system at time I, and c encapsulates the text-conee next-conditioning information. Under these conditions, the following identity holds:
the linear equation can be transformed as:
-ΣV
(15)
Considering U*U = I and denoting X' = V*X, we have ΣX' = O. Because A ~ O, all of its singular values approximately equals to D. Considering the floating-point precision we need, ΣX' = 0 could be regarded as:
60
E (91) - Ei( = VI- āi (E- (Z,,,,,) (10)
0
vār
rr
Proof. Denote Zo (o) and zu as zu after adding noise and denoising. Therefore, have
X' = 0,
0
0
0
0
& (g)) ()) & (Di (zi)) Zo zi- Zo. (11)
where D is the decoder of VAE.
(16)
Given an initial data point Zo. the forward process in a diffusion model adds noise to the data step by step. The expression for 2₁ at specific timestep & can be written as:
where is the number of non-zero singular values. There- fore, there exists an Z' E RR* with rank min(m, n) -r. When is & min(m, n)) Z' is full rank, i.e, Z = VZ' is an approximate full-rank solution to the linear equation AX = 0.
Zt
VI dGE.
(12)
To denoise 2₁ and recover an estimate of the original data Zo, the reverse process is used. A neural network is trained to predict the noise added Zg. The denoised data zi can be expressed as:
Lemma (Singular Value Decomposition). Any ma- trix can be decomposed into the product of three matrices: A UEV" where U and V are orthogo- nal matrices, Σ is a diagonal matrix with non-negative singular values of A on the diagonal, and V is the conjugate transpose of V.
2 = Vāl (2: VI oico (24,t,c))
(13)
Therefore, we have:
Proof. Consider a matrix A E RmX", The matrix A A is therefore symmetric and positive semi-definite which means the matrix is diagonalizable with an eigendecompo- sition of the form:
El (g1) (l) 2/g -20
1 (zr- V1 - 0:E0 (Ze, c))
Var
1 (Varza + v1 - asE v1 Cree (eeeu(er,,,)) (,) zo
Vot
A = VAV
Σ
Avay = (o.)) v.v, (17)
VI - at. (c- Eo (2r,,,c))
1=1
1=1
Vos
where V is an orthonormal matrix whose columns are the eigenvectors of A A.
(14)
The Eq. 10 is proved.
We have defined the singular value a; as the square root of the i-th eigenvalue; we know we can take the square root of our eigenvalues because positive semi-definite matrices can be equivalently characterized as matrices with non-negative cigenvalues.
Lemma 2. Consider the equation AX- O, where is a matrix. If approximately equals to zero matrix, i.e., A 2 O, then there exists an approximate full-rank solution to the equation.
For the i-th sigenvector-eigenval pair, we have
Proof. Consider a matrix A E R According to Lemma 3, there exists orthogonal matrices U E mm
A Av, = (a.)) vi-
(18)
12

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Traneffrre the vover suoset owe the bucling olien plane osherworiiiy plante creatune.
Ture the coental souse into forting medleval market raulle soorket stalls Biing
Reveeggine the oliffrinle view bamted forest filled glosing frova magical conutures.
Slift the voue winter wonderiand, with cidff fromm owe ior skolers.
a nature
Picture the parvot painted style of Van Gogh, Aös Storry Night.
Drow the parrot participating in a colorful carnival parade.
Show the parvot soaring over futuristic citysaaee.
Morph the parrot into a graffit art piece on an urhan wall.
(b) animal
Turn the pool table into magicaal holographic strategy bourd.
Convert the pool room inst un icy, crystailine cave with sparkling walls and cool blue huce.
Trunsform the modere pool uncient medieval lavem,
Convert the pool table into an intricate clockwork gaming table from stcumponk eru.
(c) indooT
Figure 10: Illustration of prompts and corresponding generated images for different subjects our dataset. Our dataset comprehensively includes various subjects found in the real world.
Define new vector u, such that,
I am doing Ianagetto-image lation. Could you think of creative prompts to translate this image different ones? Keep them cre- ative, and only return 20 different prompts.
Av; 0
(19)
+
This construction enables as unit cigenvector of AA Now let an n x n matrix because AA" is n x n where the i-th column is v;; let be an m X m matrix because v, is an m-vector where the i-th column is
The script for requesting GPT-4o generate 20 different prompts for each original image.
Figure 11:
u; and let Σ be a diagonal matrix whose i-th element is express the relationships we have far in
0g Then matrix form as:
(1) Suppose A m has left inverse B Raxm such that BA I, Because the I, is of rank n, the matrix AB must have rank Considering the inequality:
U= AVE-!
UE = AV,
(20)
n rank (BA) S min(rank(A)) (ankk rank(B))
(21)
A UΣV*
I Tran I min (m, n)
where the fact that VV* / and Σ 1 is diagonal matrix where the i-th value is the reciprocal of o,
we have rank(A) = n, i.e., A has full rank.
(2) Suppose E Rxxn has full rank, i.e., rank(A) min(m, n). We have the rows A are linearly independent, and thus there exists x matrix C such that CA I,- That means C is left inverse of A.
Lemma 4. A matrix has a left inverse if and only if has full rank.
Proof. To prove Lemma 4, we must demonstrate two direc- tions: a matrix A has a left inverse, then it must have full rank, and conversely, if matrix has full rank, then it has a left inverse.
B. Implementation of GPT-4o
As shown in Fig. 11, we request GPT-4o to generate 20 different prompts for each original image.
13

---

Origin Identification for Text-Guided Image-to-Iman Diffusion Models
Table 8: The influence of editing strengths on the performance of our method.
SD2 T
SDXL
OpDa
CoXL 1
Kan3
SD3 1
Kolor ?
Strength
mAP
Acc
mAP
mAP
Acc
mAP
Acc
mAP
Acc
mAP
Acc
mAP
Acc
100.0
100.0
100.0
100.0
100.0
99.9
100.0
100.0
100.0
100.0
99.9
99.9
0.2
100.0
100.0
99.9
99.9
100.0
100.0
99.9
99.9
99.9
99.9
100.0
99.9
99.8
99.8
0.3
100.0
100.0
99.9
99.8
99.9
99.9
99.9
99.8
99.7
99.7
99.9
99.9
99.6
99.5
0.4
99.9
99.9
99.8
99.7
99.9
99.8
99.8
99.7
99.2
99.1
99.9
99.8
98.9
98.6
0.5
99.9
99.8
99.5
99.3
99.6
99.4
99.4
99.2
97.6
97.1
99.7
99.6
97.4
96.9
0.6
99.8
99.7
99.1
99.0
98.2
97.8
97.9
97.5
85.7
83.3
99.4
99.2
92.1
90.8
0.7
99.2
99.0
97.9
97.5
87.3
85.3
89.3
87.7
61.9
57.2
98.5
98.1
90.3
88.8
0.8
97.5
97.1
81.5
78.8
49.5
45.4
61.0
57.1
14.5
11.4
85.7
82.9
70.9
67.4
0.9
88.8
86.6
68.1
63.9
13.2
10.7
19.5
16.5
2.1
1.5
30.1
25.6
24.5
20.5
1.0
43.2
37.7
15.7
1.8
1.2
2.6
1.8
0.0
0.0
0.0
0.0
2.2
1.5
Stable Diffusion Strongth110
Stable Diffusion XL Strengh-1..
OpenDalle Strengh-02.
ColorfuIXL Streggh--0.8
Kandinsky-3 Strongth -0.
Stable Diffusion 3 Strengdb
Kolors
Strengoh-0.9
Query
Origin
Figure 12: Generations with large editing strengths: the queries are visually very dissimilar to the origins.
C. Prompt and Generation Examples
Quecy
In Fig. 10, we present several prompts with their corre- sponding generated images from our dataset, OriPID. The dataset comprehensively covers wide range of subjects commonly found in real-world scenarios, such natural sceneries, cultural architectures, lively animals, luxuriant plants, artistic paintings, and indoor items. It is important to note that the training set, for each original image, OriPID contains 20 prompts with corresponding generated images, and for illustration, we only show of them in Fig. 10.
False match
Origin
D. Influence of Editing Strengths
Table shows how the proposed method performs under different editing strengths. We observe that although the training and testing images come from different diffusion models with varying editing strengths, the performance of method remains consistently high across most editing strengths. It is important to note that: (1) strength means it's almost like generating from pure noise, which approximately equivalent to text-to-image generation. Therefore, it is reasonable that we cannot find the origins that case; (2) Fig. 12 gives some examples of strengths where our method fails. These queries are indeed very visu- ally dissimilar with the origins; and (3) We do not change the training editing strength for Stable Diffusion 2 while varying test editing strength. That means our method is also generalizable across varying editing strengths.
Figure 13: This illustration shows failure cases predicted by our method. We have identified that our model may fail when encountering hard negative samples.
E. Failure Cases and Potential Directions
Failure cases. As shown in Fig. 13, we observe that our model may fail when negative samples are too visu- ally similar the queries. This hard negative problem is reasonable because our model a VAE to compress high- dith-siinniio inputs into lower-dimensional latent rep- resentation. This compression tends to smooth out subtle local details, causing the model to lose critical fine-grained distinctions between similar yet different instances. Further- more, the imposed prior encourages a uniform distribution in the latent space, forcing nuanced features from distinct
14

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Table 9: The performance of publicly available models on 7 different diffusion models.
SD2
SDXL f
OpDa f mAP
CoXL mAP
Kan3
SD3 T
Kolor
Method
mAP
Acc
mAP
Acc
Acc
Acc
mAP
Acc
mAP
Acc
mAP
Acc
Swin-B
3.1
2.0
2.9
1.9
4.1
2.9
4.2
3.1
6,8
4.7
2.9
1.9
3.0
2.0
Supervised
ResNet-50
3.8
2.6
3.1
2.0
5.3
3.7
4.5
3.2
8.1
5.7
3.4
2.0
3.4
2.2
Pre-trained
ConvNeXt
3.5
2.1
3.3
2.2
4.7
3.3
5.0
3.5
8.4
6.2
3.5
2.4
3.6
2.6
Models
EfficientNet
2.9
1.9
2.9
2.0
4.9
3.4
5.4
3.9
8.7
6.5
3.3
2.2
4.1
3.0
ViT-B
4.1
2.8
4.5
3.1
7.2
5.5
6.7
5.0
11.2
8.7
4.1
2.8
5.6
4.3
SimSiam
1.5
1.0
1.2
0.7
1.8
1.7
1.0
3.1
1.9
1.5
0.8
1.4
0.8
Self-
MoCov3
1.4
0.8
1.5
0.9
2.4
1.3
2.2
1.3
3.8
2.4
1.9
1.1
1.6
1.0
supervised
DINO<2
2.6
1.6
2.7
1.7
4.6
3.0
5.5
3.6
8.4
5.9
2.9
1.9
3.6
2.6
Learning
MAE
14.9
11.4
10.0
8.0
13.1
10.5
8.1
6.4
17.6
14.3
11.2
8.5
6.5
5.1
Models
SimCLR
6.0
4.2
7.0
5.2
13.5
10.6
13.0
10.1
23.7
19.3
7.3
12.0
8.8
6.7
CLIP
2.6
1.7
2.1
1.4
3.1
2.1
3.2
2.0
4.2
2.7
2.5
1.6
2.1
0.7
Vision- language Models
SLIP
5.6
3.8
3.5
2.3
5.8
4.0
4.9
3.3
9.1
6.7
5.4
3.5
3.8
2.5
ZeroVL
5.2
3.5
4.4
2.9
6.4
4.4
4.5
3.2
9.8
6.9
4.7
3.0
4.3
3.0
BLIP
6.8
4.8
6.5
4.5
9.9
7.0
8.7
6.3
13.8
10.2
6.0
3.9
6.4
4.5
ASL
2.3
1.7
3.0
2.3
5.6
4.4
5.7
4.6
10.3
8.7
2.7
2.1
3.7
2.9
Image Copy
CNNCL
4.0
2.9
4.2
3.2
8.3
6.7
5.7
4.5
12.2
9.9
3.7
2.7
6.3
5.0
Detection
BoT
6.6
4.9
6.1
4.4
10.4
8.2
12.5
10.2
20.6
16.8
7.4
5.4
9.3
7.3
Models
SSCD
9.7
7.7
8.7
6,8
16.4
14.0
18.1
15.6
28.
24.6
9.0
6.8
14.1
11.9
AnyPattem
17.6
14.3
18.5
15.7
33.0
29.2
37.8
34.0
48.0
43.9
18.2
15.0
30.7
27.5
Table 10: The performance of our trained models on 7 different diffusion models. Note that these models are trained on images generated by SD2 and tested on images from multiple models.
SD2 1 Acc
SDXL f
OpDa f Acc
CoXL 1 Acc
Kan3 f
SD3 1
Kolor f
Method
mAP
mAP
Acc
mAP
mAp
mAP
Acc
mAP
Acc
mAP
Acc
Similarity-
Circle loss
70.4
64.3
56.2
50.1
56.5
51.8
41.6
37.0
60.0
53.8
65.6
59.3
43.5
39.2
based
SoftMax
82.7
78.3
62.4
56.5
58.3
53.0
37.3
32.2
52.5
46.0
75.9
70.2
43.6
38.7
Models
CosFace
87.1
83.2
63.7
58.2
56.7
51.7
30.5
25.2
47.5
40.6
71.5
65.5
43.0
38.0
General-
IBN-Net
88.6
85.1
65.7
60.1
59.4
54.2
33.3
28.3
49.8
42.8
74.0
68.3
45.4
40.5
izable
TransMatcher
65.6
60.3
60.6
55.8
67.9
63.6
61.7
57.4
68.9
64.2
64.7
59.2
67.9
63.9
Models
QAConv-GS
78.8
74.9
71.6
67.5
77.4
74.3
73.6
70.5
75.2
71.2
77.3
73.6
79.5
76.9
VAE Embed.
51.0
47.0
38.3
33.8
42.3
38.6
51.6
48.8
54.7
50.4
47.7
42.9
46.9
43.6
Ours
Lincar Trans.
88.8
86.6
81.5
78.8 8
87.3
85.3
89.3
87.7
85. 7
83.3
85.7
82.9
90.3
88.8
Upper 88.8
86.6
84.9
82.4
90.8
89.2
93.1
91.9
95.4
94.3
93.7
92.0
94.0
92.8
instances into overlapping latent representations. These factors reduce the model's capability to differentiate hard negatives from true positives.
directly lesting publicly available models on the OriPID test dataset; and (2) Table 10 shows the results from testing mod- els that we trained on the OriPID training dataset, which contains only images generated by Stable Diffusion 2.
The hard negative problem has been studied in the Image Copy Detection (ICD) community, as exemplified by ASL (Wang et al., 2023). It learns to assign a larger norm to the deep features of images that contain more content or information. However, this method can- not be directly used in our scenario because the query and reference here do not have simple relationship in terms of information amount. Nevertheless, it offers promis- ing research direction from the perspective of information. Specifically, on one hand, the noise-adding and denoising processes result in a loss of information, while on the other hand, the text introduces new information into the output.
Potential directions.
G. Limitations and Future Works
Limitations. Although the paradigm analyzed in the main paper the simplest approach for text-guided image-to- image translation and serves as the default mode in the AutoPipell ietooippeinneFor on or Image2Image of diffusers, we also observe the existence of alternative paradigms, such as In- StructPix2Pix (Brooks et al., 2023), IP-Adapter (Ye et al., 2023), EDICT (Wallace et al., 2023), and Plug-and- (Tu- manyan et al., 2023). While these paradigms lie beyond our theoretical guarantees, we can still analyze them experimen- tally, as demonstrated in Table 11. Interestingly, we find that (1) our method generalizes well InstructPix2Pix, EDICT and Plug-and-Play, which still use VAE encoder to embed the original images; and (2) we fail on IP- IP-Adatter, which
F. Complete Experiments for 7 Models
We provide two types of complete experiments for seven dif- ferent diffusion models: (1) Table 9 presents the results from
15

---

Origin Identification for Text-Guided Image-to-Image Diffusion Models
Table 11: The generalization results on InstructPix2Pix (Brooks et al., 2023), IP-Adapter (Ye et al., 2023), EDICT (Wallace et al., 2023), and Plug-and-PPe and-Play (Tumanyan et al., 2023).
InstructP2P f
IP-Adapter
EDICT f
Plug-an -nd--aaddPaal f
Method
mAP
Acc
mAP
Acc
mAP
Acc
mAP
Ace
VAE Embed.
68.2
67.1
0.2
0.1
60.4
55.4
99.4
99.1
Ours
Linear Trans. VAE
80.7
79.2
0.4
0.2
89.0
86.6
99.8
99.7
uses CLIP for encoding. We also try the linear transformed CLIP embedding, but it still fails generalize (36.6% mAP and 27 87. Acc). Based these experiments, we conclude with hypothesis about the upper limit of our method:
Hypothesis 1. Following Theorem I, consider a dif- ferent well-trained diffusion model F3 and text- guided image-to-image functionability achieved with VAE-encoded original images. The matrix W can be generalized such that for any original image and its translation 93. we have:
(gs) W = E(() W.
(22)
Future Works. Future works may focus on (1) providing a theoretical proof for Hypothesis I, and (2) developing new generalization methods for text-guided image-to-image based on CLIP encodings.
16


Reviews:
[{"review_id": "GKOlwTfERl", "reviewer": "Reviewer_uztF", "summary": "This paper introduces the ''Origin Identification'' task for text-guided image-to-image diffusion models, aiming to retrieve the original image of a given modified image generated by diffusion models. The motivation for this task stems from security concerns, including misinformation, copyright infringement, and content tracing evasion. Moreover, this paper proposes a novel dataset, containing 100,000 original images with 20 guided prompts per image and 2,000,000 training images, and designs a theoretically guaranteed identification method.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts in reviewing our paper. We are encouraged that you find: (1) this paper's claims are **well-supported**; (2) the proposed methods and evaluation criteria are **well-aligned** with the origin identification task; (3) the experimental design is generally **well-structured** with **well-justified** evaluation metrics; (4) the supplementary is **well-structured**; and (5) this paper presents a **well-executed** study. We address your questions below and will add these into the final version.\n\n**Q1. Testing on more diverse diffusion models (e.g., InstructPix2Pix, IP-Adapter) would strengthen this claim. The authors only tested on VAE-based diffusion models.**\n\nA1. Thank you for your kind reminder. As shown in the Appendix (**Table 9** and **Sec. G**), we have evaluated our model’s performance on InstructPix2Pix and IP-Adapter, which use VAE and CLIP to encode original images, respectively. The experiments show that our method successfully generalizes to InstructPix2Pix but fails on IP-Adapter.\n\nBased on these experimental results and the theoretical analysis in the main paper, we argue that the **generalization boundary** of our method is ***whether the generated image is conditioned on a VAE-encoded original image***. Notably, this VAE can vary in *architecture* and *parameters*—such as *Stable Diffusion*, *Kolors*, and *Flux*, and the conditioning schemes can also differ—such as those used in *SDEdit* and *InstructPix2Pix* (shown in our paper), or *Prompt-to-Prompt* and *Plug-and-Play* (see experiments in the **Q2** of Reviewer **gjdS**).\n\nAlthough this generalization boundary does not cover all possible scenarios, our method remains **practically valuable** because: (1) most current image-to-image methods indeed utilize a VAE to encode original images; and (2) as shown in Table 9, in fact, no existing methods succeed on IP-Adapter.\n\nIn conclusion, our method is **currently the most effective one with substantially superior performance**. Nevertheless, we acknowledge the importance of overcoming this generalization boundary and consider it as future work.\n\n**Q2. This paper does not compare linear transformation against other transformations.**\n\nA2. Thank you for your insightful question. As shown in **Fig. 9** of the main paper and **Sec. F** of the Appendix, we have discussed the transformations of **multilayer perceptrons (MLPs) with activation functions**. We observe that this case leads to an overfitting problem.\n\nHere, we add two alternative architectures: **a single convolutional layer** and **a multi-head attention layer**. The below experiments show that: (1) likely due to underfitting, the convolutional layer results in a performance drop; and (2) although the multi-head attention layer marginally improves performance on seen images, its performance on unseen images falls behind our method, due to overfitting.\n|Method|mAP (Seen)|Acc (Seen)|mAP (Unseen)|Acc (Unseen)|\n|-|-|-|-|-|\n|Embeddings of VAE|51.0|47.0|46.9|43.0|\n|Convolution|37.4|33.8|32.5|29.6|\n|Attention|89.0|87.2|80.7|78.2|\n|Linear Transformation|88.8|86.6|86.6|84.5|\n\n**Q3. Introducing hard negative mining.**\n\nA2. Thank you for your valuable suggestion. Following it, we combine our currently-used CosFace with a hard mining triplet loss. However, as shown in the table below, this approach does not bring performance improvement. This result is reasonable, as the original CosFace paper suggests that once we aim to train a large margin between classes, additional hard negative mining becomes unnecessary.\n|Method|mAP (Seen)|Acc (Seen)|mAP (Unseen)|Acc (Unseen)|\n|-|-|-|-|-|\n|With|88.4|86.4|86.8|84.9|\n|Without|88.8 | 86.6|86.6|84.5|\n\n**Q4. No evaluation on image cropping, resizing, or watermarking.**\n\nA4. Thank you for your valuable suggestion. Following it, we conduct experiments involving *image cropping*, *resizing*, and *watermarking*. As shown in https://huggingface.co/datasets/ICML2025Rebuttal/ICML2025_Rebuttal/resolve/main/mod.pdf, our method is **relatively robust** against these modifications.\n\n**Q5. This paper does not deeply analyze failure cases.**\n\nA5. Thanks for this insightful suggestion. We provide an analysis here: our model uses a VAE to compress high-dimensional inputs into a lower-dimensional latent representation. This compression tends to smooth out subtle local details, causing the model to lose critical fine-grained distinctions between similar yet different instances. Furthermore, the imposed prior encourages a uniform distribution in the latent space, forcing nuanced features from distinct instances into overlapping latent representations. These factors reduce the model’s capability to differentiate hard negatives from true positives.\n\n**Q6. Are the proposed datasets and codes publicly available?**\n\nA6. Yes! All the proposed datasets (including training, query, and gallery images) and all code (including training, testing, and dataset-generation code) will be made publicly available.", "claims_and_evidence": "Overall, this paper's claims are well-supported by theoretical arguments and experimental evidence. However, some claims could benefit from further clarification or additional empirical analysis:\n\n1. Testing on more diverse diffusion models (e.g., InstructPix2Pix, IP-Adapter) would strengthen this claim.\n\n2. This paper claims that the linear transformation approach is the best way to generalize across models, but it does not compare against other potential transformations (e.g., non-linear embeddings).", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-aligned with the origin identification task. The OriPID dataset, linear transformation method, and evaluation metrics effectively address the task.", "theoretical_claims": "The paper makes two major theoretical claims (Existence of a Linear Transformation that Minimizes the Distance Between Original and Generated Image Embeddings, Generalizability of the Linear Transformation Across Diffusion Models), each supported by formal proofs.", "experimental_designs_or_analyses": "The experimental design is generally well-structured, with appropriate datasets, strong baseline comparisons, and well-justified evaluation metrics. However, some aspects could be improved:\n\n1. Introducing hard negative mining (i.e., selecting the most confusing negatives to refine the model) and reporting error analysis on top failure cases would improve the quality of this paper.\n\n2. Add adversarial robustness evaluations or test against image compression and resizing distortions commonly seen in social media uploads.", "relation_to_broader_scientific_literature": "This paper's contributions align with several key areas in AIGC, AI security, particularly in image provenance, diffusion models, and content attribution.", "essential_references_not_discussed": "This paper does a good job of covering related literature in image retrieval, diffusion models, and AI-generated content detection.", "other_strengths_and_weaknesses": "This paper presents a well-executed study on origin identification for text-guided diffusion models, with notable strengths in problem formulation, theoretical grounding, and practical evaluation. However, some limitations remain in generalization, robustness, and interpretability:\n\n1.  The authors only tested on VAE-based diffusion models, it would be better to discuss the generalizationability of the proposed method for the future diffusion models, which may use different encoders.\n\n2. The proposed method is only tested on Gaussian blur and JPEG compression, but real-world adversaries may apply more advanced modifications.\n\n3. No evaluation on image cropping, resizing, or watermarking.\n\n4. No insight into why some transformations fail. This paper does not deeply analyze failure cases beyond mentioning hard negatives.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "Are the proposed datasets and codes publicly available to the academic community?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eKgYWxHCHb", "reviewer": "Reviewer_PzDk", "summary": "- This paper introduces a new problem, \"origin identification\", for text‐guided image‐to‐image diffusion models, with the goal of retrieval the original image given a query image that was transformed by a text‐conditioned diffusion model.\n- The paper proposes a new dataset OriPID, containing original images, text prompts, and query images produced by seven popular diffusion models.\n- The paper also provides a novel retrieval‐based method learns a single linear transformation so that an original and its generated variant lie close in the transformed embedding space.\n- The paper includes theoretical arguments showing such a linear transformation should exist, and it should generalize to unseen diffusion models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts in reviewing our paper. We are encouraged that you find our work (1) provides a **large-scale** and **carefully curated** benchmark, (2) proposes a **novel** retrieval-based method, (3) includes **theoretical** arguments, (4) is **intuitive** and **make sense**, and (5) is **empirically superior** to baselines. We address your questions below and will incorporate all your suggestions into the final version of our paper.\n\n**Q1. The proposed dataset does not containing the editing strength, and the paper does not show how will the proposed method perform with different editing strength.**\n\nA1. Thanks for your kind reminders. During testing, the editing strengths for Stable Diffusion 2, Stable Diffusion XL, OpenDalle, ColorfulXL, Kandinsky-3, Stable Diffusion 3, and Kolors, are 0.9, 0.8, 0.7, 0.7, 0.6, 0.8, and 0.7, respectively. The editing strengths used in testing are manually set to prevent significant visual differences between the generated images and the original ones. During training, the editing strength for Stable Diffusion 2 is 0.9.\n\nThe table below shows how the proposed method performs under different editing strengths. We observe that although the training and testing images come from ***different diffusion models*** with ***varying editing strengths***, the performance of our method remains consistently **high** across **most** editing strengths. \n\nIt is important to note that: \n- `strength = 1` means it's almost like generating from pure noise, which is approximately equivalent to text-to-image generation. Therefore, it is reasonable that we cannot find the origins in that case. \n- As shown in https://huggingface.co/datasets/ICML2025Rebuttal/ICML2025_Rebuttal/resolve/main/fail_example.pdf, we give some examples of strengths where our method fails. These queries are indeed very visually dissimilar with the origins.\n- We do **not** change the training editing strength for Stable Diffusion 2 while varying test editing strength. That means our method is also generalizable across varying editing strengths. \n\n\n| mAP | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |1.0 |\n|------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|\n|   Stable Diffusion 2  | 100.0 |  100.0   |  100.0   |   99.9  | 99.9  |  99.8 | 99.2    | 97.5    |  88.8   |43.2 |\n| Stable Diffusion XL   | 99.9 | 99.9 | 99.9| 99.8 |  99.5  |  99.1|  97.9 | 81.5 |  68.1  |19.3|\n| OpenDalle   |  100.0   |  100.0   |  99.9   |  99.9   |  99.6   |  98.2   |   87.3  |   49.5  |   13.2  | 1.8|\n| ColorfulXL    |  100.0  |  99.9  |   99.9  |  99. 8  |   99.4  |  97.9   |   89.3   |  61.0   |   19.5  |2.6 |\n| Kandinsky-3    |   100.0  |  99.9   |   99.7  |  99.2   |   97.6  |  85.7   |   61.9  |  14.5   |  2.1   | 0.0|\n| Stable Diffusion 3    |  100.0   | 100.0    | 99.9    |  99.9   |  99.7   |   99.4  |  98.5   |  85.7   |  30.1   |0.0|\n| Kolors    |   99.9  |   99.8  |   99.6  |  98.9   |   97.4  |   92.1 | 90.3   |  70.9   |  24.5   |  2.2   | \n\n| Acc | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |1.0 |\n|------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|\n|   Stable Diffusion 2  | 100.0 |  100.0   |  100.0   |  99.9   |  99.8   |  99.7   |  99.0   |  97.1 |  86.6  |37.7 |\n| Stable Diffusion XL   | 99.9 |99.9| 99.8 |  99.7  |  99.3  |  99.0 | 97.5 |  78.8 |   63.9  |15.7|\n| OpenDalle   |  100.0   |   100.0  |   99.9  |  99.8   |   99.4  |   97.8  |  85.3   |   45.4  |  10.7   |1.2 |\n| ColorfulXL    |   99.9  |   99.9  |    99.8 |   99.7  | 99.2    |   97.5  |   87.7   |   57.1  | 16.5    | 1.8|\n| Kandinsky-3    |  100.0   |   99.9  |  99.7   |  99.1   |  97.1   |  83.3   |  57.2   |  11.4   |  1.5   |0.0 |\n| Stable Diffusion 3    |   100.0  |  99.9   |   99.9  |   99.8  |    99.6 |  99.2   |  98.1   |  82.9    |  25.6   | 0.0|\n| Kolors    |  99.9   |  99.8   |   99.5  |  98.6   |  96.9| 90.8   |  88.8   |  67.4   | 20.5    |   1.5  |", "claims_and_evidence": "1. **Claim:** The paper claims that for a well‐trained text‐to‐image diffusion model and its VAE encoder, one can learn a single linear transform that maps each generated‐image embedding close to its original‐image embedding. **Evidence:** Theorem 1 show the derivations and proves it. Empirical results in Table. 2 also validates it.\n2. **Claim:** The same linear transform can work well for images generated by other unseen diffusion models. **Evidence:** Theorem 2 show the derivations and proves it. Empirical results in Fig. 9 also validates it.", "methods_and_evaluation_criteria": "1. The proposed method simply includes VAE embeddings, linear transformation.\n2. For the proposed origin identification task, the paper uses mAP and accuracy for evaluation.\n3. The proposed methods and evaluation criteria make sense and intuitive.", "theoretical_claims": "See the Claims And Evidence section.", "experimental_designs_or_analyses": "1. The paper conducted extensive experiments on the proposed OriPID dataset, and compare the transformation matrix with baseline methods, including classification models, self‐supervised models, vision‐language models, and image copy detection models.\n2. The paper ablate the proposed method by using different VAE encoders, loss functions, and different matrix rank, showing the propose method is insensitive to VAE encoder, and MLP with activation leads to overfitting.", "relation_to_broader_scientific_literature": "1. The proposed task is related to image copy detection, domain generalization, and text-guided image editing.\n2. The paper underscores the issue of manipulated images for malicious or illegal ends, aligning with concerns in generative‐model detection.", "essential_references_not_discussed": "No essential references missed.", "other_strengths_and_weaknesses": "**Strengths**\n1. The paper offers a large‐scale benchmark (OriPID), carefully curated, which should help standardize evaluations in this new domain.\n2. The proposed solution is simple yet theoretically motivated and empirically superior to baselines.\n\n**Weaknesses**\n1. The proposed dataset does not containing the editing strength, and the paper does not show how will the proposed method perform with different editing strength.", "other_comments_or_suggestions": "See previous sections", "questions_for_authors": "See previous sections", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iZSIULsvWk", "reviewer": "Reviewer_VEsJ", "summary": "This paper introduces the Origin Identification for Text-Guided Image-to-Image Diffusion Models (ID^2) task, aiming to retrieve the original image of a given translated query. The paper highlights the risks of misuse, including misinformation, copyright infringement, and evading content tracing. A key contribution is OriPID, a dataset containing a large-scale reference set of images and guided prompts, designed to test the generalizability of ID2 models across different diffusion models. The paper also presents a theoretically guaranteed method that minimizes the distance between Variational Autoencoder embeddings of generated samples and their origins through a learned linear transformation, demonstrating generalizability across different diffusion models. Experimental results show a +31.6% mAP improvement over similarity-based retrieval methods.\n\n## update after rebuttal\n\nAll of my questions have been fully addressed, and I appreciate the thoughtful and detailed responses. I maintain my positive rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts in reviewing our paper. We are encouraged that you find: (1) the proposed linear transformation approach is **simple**, **efficient**, and **theoretically grounded**; (2) OriPID dataset provides a **strong benchmark** for **future work** on ID$^2$; (3) Extensive experiments demonstrate **generalization** across **multiple diffusion models.** We address your questions below and will incorporate all your suggestions into the final version of our paper.\n\n**Q1. The prerformance on other diffusion-based models, e.g., FLUX.1-dev, is not explored.**\n\nA1. Thank you for this valuable suggestion. Accordingly, we evaluate our proposed method on images generated by FLUX.1-dev. The experiments below show that our method generalizes well to FLUX.1-dev.\n|Methods|mAP|Acc| \n|---|---|---| \n|Circle loss|71.2|67.1|\n|SoftMax|65.1|61.7|\n|CosFace|68.9|64.2| \n|IBN-Net|72.2|68.5|\n|TransMatcher|76.9|72.6|\n|QAConv-GS|76.8|71.9|\n|VAE Embed. (Ours)|52.3|47.6| \n|**Linear Trans. VAE  (Ours)**|**88.9**|**87.1**|\n\n**Q2. The paper does not include visualizations of successful/failed origin identification results, which would make the claim stronger.**\n\nA2. Thank you for this valuable suggestion. Accordingly, we visualize several successful origin identification results, available at https://huggingface.co/datasets/ICML2025Rebuttal/ICML2025_Rebuttal/resolve/main/success.pdf. Our observations indicate that the origin of queries—spanning various topics and generated by different diffusion models—can be effectively traced. The failure cases are available in the Appendix (Section E).\n\n**Q3. Current T2I and I2I models also contributed by auto-regressive models. However, this paper only works on diffusion-based methods.**\n\nA3. Thank you for your valuable comment. We acknowledge these autoregression advancements and the current scope of our work is indeed focused on diffusion-based methods. This is because diffusion models remain a cornerstone in both research and industry due to their proven reliability and high-quality results. As a result, the dataset, method, and theory we introduce are directly applicable to practical scenarios and thus useful.  Nevertheless, we agree that exploring auto-regressive approaches is an important future direction, and we plan to investigate them in our subsequent work.\n\n**Q4. Limited real-world validation: The dataset is synthetic, and real-world image alterations (e.g., Photoshop edits) are not explored.**\n\nA4. Thanks for this valuable suggestion. Accordingly, we evaluate our method on a new dataset, SEED-Data-Edit [1], which contains 52k image editing samples. These samples were collected from **amateur photographers** who posted their images along with editing requests. **Photoshop** experts then fulfilled these requests, providing the edited images as target images.\n\nExperimentally, we (1) de-duplicate to get 10,274 image pairs; and (2) treat the edited (target) images as queries and search for them within a pool consisting of their origins along with 1,000,000 distractor images. \n\nThe experiments below show that: (1) our method generalizes effectively to real-world, manually edited images; and (2) it achieves the best performance compared to all competing methods.\n\n|Methods|mAP|Acc| \n|---|---|---| \n|Circle loss|76.6|74.5|\n|SoftMax|76.2|73.2|\n|CosFace|73.1|70.1| \n|IBN-Net|75.4|72.1|\n|TransMatcher|78.3|76.4|\n|QAConv-GS|74.4|71.9|\n|VAE Embed. (Ours)|66.6|64.6| \n| **Linear Trans. VAE  (Ours)**|**86.6**|**85.5**|\n\n[1] Seed-data-edit technical report: A hybrid dataset for instructional image editing\n\n**Q5. Contribution Point 3 in introduction lists seven performance numbers but does not directly specify which numbers correspond to which models.**\n\nA5. Thanks for this suggestion. We will list them explicitly in the final version: *(2) the effectiveness of our proposed method: it achieves 88.8%, 81.5%, 87.3%, 89.3%, 85.7%, 85.7%, and 90.3% mAP, respectively, for Stable Diffusion 2, Stable Diffusion XL, OpenDalle, ColorfulXL, Kandinsky-3, Stable Diffusion 3, and Kolors.*\n\n**Q6. What happens if an image is generated by an auto-regressive model and later modified by a diffusion model?**\n\nA6. Thank you for the question. In response, we (1) generate 5,000 images using Janus-Pro-7B and modify them with ColorfulXL, and (2) use the modified images as queries to search within a pool containing the original Janus-Pro-7B outputs along with 1,000,000 distractor images.\n\nThe experimental results indicate that our method achieves **similar performance** whether the origin is a real image or one generated by an auto-regressive model.\n|Origin|mAP|Acc|\n|---|---|---|\n|Real|89.3|87.7|\n|Auto-regressive|90.5|88.4|\n\n**Q7. Will the paper release the OriPID dataset?**\n\nA7. Yes! All the proposed datasets (including training, query, and gallery images) and all code (including training, testing, and dataset-generation code) will be made publicly available to facilitate future research.", "claims_and_evidence": "The paper claims that:\n\n1. Similarity-based methods fail to generalize across diffusion models → Supported by experiments, e.g., Table 4, Section 5.2, showing a sharp drop in mAP when training and testing on different models .\n2. A linear transformation can bridge the gap in VAE embeddings across models → Proven theoretically and empirically, with experiments demonstrating improved performance.\n\nHowever, the claim of generalizability to all diffusion models lacks thorough validation, as only specific models (e.g., Stable Diffusion variants) are tested. The prerformance on other diffusion-based models, e.g., FLUX.1-dev, is not explored.", "methods_and_evaluation_criteria": "The evaluation metrics (mAP, top-1 accuracy) are standard and appropriate. The training/testing split across different models is a strong design choice to assess generalization.", "theoretical_claims": "The paper provides proofs for the existence and generalizability of a linear transformation that aligns VAE embeddings of original and modified images. The mathematical foundation appears correct.", "experimental_designs_or_analyses": "The experiments are well-structured, covering:\n\n- Performance across seven diffusion models.\n- Efficiency comparisons with similarity-based and generalization-focused methods.\n\nHowever, the paper does not include visualizations of successfully identifying the origin of a generated query, which would make the claim stronger. Visualizing retrieval cases, including both successes and failures, would significantly improve interpretability.", "relation_to_broader_scientific_literature": "The paper contributes to image modification detection within diffusion models.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "- Strength\n    - The proposed linear transformation approach is simple, efficient, and theoretically grounded.\n    - OriPID dataset provides a strong benchmark for future work on ID2.\n    - Extensive experiments demonstrate generalization across multiple diffusion models.\n- Weakness\n    - The paper does not include visualizations of successful origin identification results, which would make the claim stronger.\n    - Does not discuss commonly used diffusion-based model, FLUX.1-dev.\n    - Current T2I and I2I models also contributed by auto-regressive models, e.g., Janus-pro[A], emu3[B], VAR[C], etc. However, this paper only works on diffusion-based methods.\n    - Limited real-world validation: The dataset is synthetic, and real-world image alterations (e.g., Photoshop edits) are not explored.\n    \n    References:\n    \n    [A] Chen et al., Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. ArXiv, 2025\n    \n    [B] Wang et al., Emu3: Next-Token Prediction is All You Need. ArXiv, 2024.\n    \n    [C] Tien et al., Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction. NeurIPS 2024", "other_comments_or_suggestions": "Contribution Point 3 in introduction lists seven performance numbers but does not directly specify which numbers correspond to which models. I would suggest either list them explicitly or provide an averaged score.", "questions_for_authors": "- What happens if an image is generated by an auto-regressive model and later modified by a diffusion model? How well would ID2 handle this scenario?\n- Have the paper tested the proposed approach on real-world, manually edited images (e.g., Photoshop modifications or any random AI editing tools)? Would the method generalize to such cases?\n- Will the paper release the OriPID dataset?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "STTdnkgmBW", "reviewer": "Reviewer_gjdS", "summary": "This paper introduces a new task, identifying the original image for a generated image from text-guided image-to-image translation with diffusion models, which helps prevent the misuse of the generated content such as misinformation and copyright infringement. To deal with this problem, the authors build a dataset OriPID and propose a novel method with theoretical derivations. Although some questions remain regarding the claimed \"theoretical guarantee\" and generalization to other translation methods, the experimental analysis appears thorough and comprehensive. The paper is well-written and well-structured.\n\n##  update after rebuttal\nThe authors have addressed my concerns and I recommend accepting this paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your efforts in reviewing our paper. We are encouraged that you find: (1) Most of claims are supported by **clear** and **convincing** evidence; (2) this paper proposes an **important**, **novel**, and **interesting** task; (3) the analysis appears **thorough** and **comprehensive**; (4) the paper is **well-written**, **well-structured**, and **easy to follow**. We address your questions below and will add these into the final version.\n\n**Q1. In the proof of Lemma 1, the derivation of z_0' in Eq. 13 appears problematic.**\n\nA1. We apologize for the confusion. We acknowledge that, in practice, this process is typically iterative, refining $z_t$ over many steps. However, Eq. 13 does not bypass the chain; rather, it is a  **mathematically equivalent alternative**, as shown in [1] (See Eq. 15 and *Progressive generation*) and [2] (See Eq. 12 in https://shorturl.at/xhnd7). Furthermore, [1] also explicitly mentions that *“there is also the possibility of predicting $x_0$”*, but this gave worse empirical quality than iterative noise-prediction. \n\nIn other words, Eq. 13 is a convenient analytical step; practically, the preference for multi-step denoising is due to empirical gains from refining the prediction gradually. Please also see a discussion here: https://shorturl.at/RBZER. **Nevertheless, we are happy to further discuss and revise Eq. 12 to Eq. 13.**\n\n[1] Denoising Diffusion Probabilistic Models, NeurIPS 2020\n\n[2] Improved Denoising Diffusion Probabilistic Models, ICML 2021\n\n**Q2. How do you encode the origin? Do you directly add noise to the origin latents as SDEdit? Does DDIM inversion and attention control affect results, such as in prompt-to-prompt and plug-and-play?**\n\nA2. Thank you for the insightful question! The origins are first encoded by VAE, and noise is directly added to the resulting image latents, as in SDEdit. Nevertheless, *DDIM inversion* and *attention control* do ***not challenge the generalizability of our method***. Experiments are shown below.\n\n**Experimental Setup.** Since Prompt-to-Prompt cannot edit real images, we adopt its improved version, EDICT [1], a re-formulation of the DDIM process that allows mathematically exact inversion. Specifically, we:\n- select 1,000 origins and ask GPT-4o to generate inversion prompts;\n- input the inversion prompts, origins, and guidance prompts into EDICT and Plug-and-Play to generate translations. Here, we follow their original parameters;\n- search these queries in 1,000,000 images, consisting of origins and distractor images.\n\n**Experimental Results.** The table below shows that our model, trained on **SD 2** and the **SDEdit** scheme, successfully generalizes to these **new schemes**. Again, these experiments **validate** our Hypothesis 1, i.e., the **generalization boundary** of our method is determined by ***whether the generated image is conditioned on a VAE-encoded original image***.\n|Methods|mAP|Acc| \n|-|-|-|\n|EDICT  (SD 1.4)|89.0|86.6|\n|Plug-and-Play (SD 1.5)|99.8|99.7|\n\n[1] Edict: Exact diffusion inversion via coupled transformations, CVPR 2023\n\n**Q3.  The influence of the denoising strength. Does it still make sense to retrieve the origin if the generated image bears little resemblance to it (Fig. 10 (c) indoor)?**\n\nA3. Thank you for the insightful question! The influence of denoising strength is discussed in **Q1** by **Reviewer PzDk**. We acknowledge that it is challenging to clearly define resemblance. Nevertheless, the origin and generations in Fig. 10 (c) share similarities, such as the pool table, indoor setting, lighting placement, viewpoint, and room layout. We argue that retrieving the origins at very large strengths does not make sense, as shown by our failures: https://huggingface.co/datasets/ICML2025Rebuttal/ICML2025_Rebuttal/resolve/main/fail_example.pdf\n\n**Q4. How does CFG affect results?**\n\nA4. Thank you for the insightful question! In our paper, experiments are conducted at CFG=7.5, which is default for most diffusion models. The table below shows our method **performs well** across many commonly-used CFGs. These experiments are obtained by training on SD 2 at CFG=7.5 and testing on ColorfulXL at varying CFGs.\n|CFG|3.5|4.5|5.5|6.5|7.5|8.5|9.5|10.5|\n|-|-|-|-|-|-|-|-|-|\n|mAP|94.9|93.9|92.3|90.5|88.8|87.7|86.3|84.8|\n|Acc|94.0|92.8|91.0|89.0|87.1|85.9|84.4|82.6|\n\n**Q5. InstructPix2Pix and IP-Adapter encode the origin through VAE latents concatenation and CLIP embeddings, which is discussed in App. G. Fig. 13 (b) is a bit confusing.**\n\nA5. Thank you for the kind reminder! Fig. 13 (b) aims to provide a unified perspective on how InstructPix2Pix and IP-Adapter encode the origin. We apologize for omitting the concatenation in InstructPix2Pix. Following your advice, we will replace Fig. 13 (b) in the **App. G** with textual description in the **Related Work**.\n\n**Q6. In Fig. 10, there are 3 subjects instead of 6?**\n\nA6. Thank you for the kind reminder! You're right—we'll fix it to **3** in the final version.", "claims_and_evidence": "Most of the claims are supported by clear and convincing evidence:\n1. The proposed dataset OriPID supports the experimental evaluations of the origin identification problem, in both seen and unseen settings regarding visual discrepancy between different diffusion models as shown in Figure 2.\n2. Existing methods have difficulties in handling the proposed task. They either completely fail (pre-trained deep embedding models in Table 2), or cannot generalize to unseen scenarios (fine-tuned similarity-based methods and speciallized domain generalization methods in Table 4).\n3. The proposed method is supported with theoretical derivation and implementation details. It achieves good performance in origin identification in both seen and unseen settings as in Table 4, with advantages in generalization, efficiency and robustness. The ablation studies further demonstrate the efficacy of the proposed method.", "methods_and_evaluation_criteria": "The proposed method, linearly transforming the VAE embeddings which is learned using a metric learning loss, and the evaluation criteria in mAP and Acc makes sense.", "theoretical_claims": "In the proof of Lemma 1, the derivation of z_0' in Eq. (13) appears problematic. The estimation of noise by the trained network is a step-by-step process, meaning z_0' cannot be directly obtained by reversing Eq. (12) in a single step. This makes the reasoning in Line 235-239 for Theorem 1 questionable.", "experimental_designs_or_analyses": "I checked all the experimental designs and analyses and found no further issues.", "relation_to_broader_scientific_literature": "This paper proposes a novel task with an effective solution to address the potential misuse of text-guided image-to-image tranlsation results by diffusion models. The benchmark built in this work and the proposed method should further facilitate the exploration of generative content detection and tracing for security concerns.", "essential_references_not_discussed": "One series of related works that is missing concerns text-guided image-to-image translation using diffusion models. The scope of translation mechanisms that the proposed method can cover remains unclear. There are some aspects that I think could impact the effectiveness of the proposed method:\n1. How do you encode the original image during translation? In Line 750, \"the default mode in the AutoPipelineForImage2Image of diffusers\" respresents directly adding noise to the original image latents as in SDEdit [1]? What if you use DDIM inversion [2] as in prompt-to-prompt [3]? How dose the denoising strength affect the results? If the denoising strength is high, the translation results could largely rely on the text prompts. Does the method still performs well with denoising strength 1.0? Does it still make sense to retrieve the original image if the generated image bears little resemblance to it (Figure 10 (c) indoor)? How does the CFG affect the results?\n2. InstructPix2Pix and IP-Adapter encode the original image through VAE latents concatenation and CLIP embeddings, which is discussed in Appendix G. Figure 13 (b) is a bit confusing.\n3. Does attention control affect the results? Such as in prompt-to-prompt [3] and plug-and-play [4]?\n\n[1] Meng, Chenlin, et al. \"SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations.\" International Conference on Learning Representations, 2022.\n\n[2] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising Diffusion Implicit Models.\" International Conference on Learning Representations, 2021.\n\n[3] Hertz, Amir, et al. \"Prompt-to-Prompt Image Editing with Cross-Attention Control.\" International Conference on Learning Representations, 2023.\n\n[4] Tumanyan, Narek, et al. \"Plug-and-play diffusion features for text-driven image-to-image translation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.", "other_strengths_and_weaknesses": "Strengths:\n1. This paper proposes an important and interesting task.\n2. Most of the claims are supported with evidence as mentioned above.\n3. The paper is well-written and easy to follow.\n\nWeaknesses:\n1. There are questions in the theoretical derivation.\n2. Some related works are missing with sufficient discussion/comparisons, which could impact the scope of this work.", "other_comments_or_suggestions": "In the caption of Figure 10, there are 3 different subjects instead of 6?", "questions_for_authors": "Please refer to the weaknesses mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers acknowledge the significance of the target problem, find the proposed method to be well-grounded in theory, and consider both the theoretical analysis and empirical results to provide strong support for the paper’s claims. While there are some concerns regarding potential overclaims about generalizability and the superiority of the linear transformation—along with the lack of failure case analysis, detailed performance evaluation, and questions about the soundness of certain derivations—these issues have been adequately addressed in the rebuttal. All reviewers have given positive final ratings. The Area Chair agrees with the reviewers and considers the paper to be solid, provided the authors appropriately incorporate the clarifications and additional content from the rebuttal into the final version.
Decision: Accept (poster)