SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We propose Algorithm 1 to determine if a specific sample belongs to a given model. The input of Algorithm 1 is the examined data  $\pmb{x}$  and the given model  $\mathcal{M}$ . The output of this algorithm is the inference results, i.e., belonging or non-belonging. In line 3, we use the given model to generate  $N$  (i.e., 100 by default in this paper) images with randomly sampled inputs and calculate the mean value  $(\mu)$  and standard deviation  $(\sigma)$  for the calibrated reconstruction loss on the generated belonging samples. This step can be done offline, i.e., it only needs to be performed once for each model. In line 5, we calculate the calibrated reconstruction loss of the examined image (Eq. 2), the reconstruction loss is computed via gradient descent optimizer (Adam [55] by default in this paper). In line 7, we determine if the examined image  $\pmb{x}$  belongs to the model  $\mathcal{M}$  or not by conducting the Grubbs' Hypothesis Testing [54] (§ 4.3). The given image is flagged as a belonging of the given model if the corresponding hypothesis is accepted.
In this section, we first introduce the setup of the experiments (§ 5.1). We evaluate the effectiveness of RONAN (§ 5.2) and provide a case study on Stable Diffusion v2 model [3] (§ 5.3). We then conduct ablation studies in § 5.4, and discuss the comparison to existing reconstruction based attribution methods in § 5.5. The discussion about the efficiency and robustness against image editing can be found in the Appendix.
Our method is implemented with Python 3.8 and PyTorch 1.11. We conducted all experiments on a Ubuntu 20.04 server equipped with six Quadro RTX 6000 GPUs.
Models. Eleven different models are included in the experiments: DCGAN [1], VAE [6], StyleGAN2-ADA [10], DDIM [2], DDPM [7], TransGAN [56], StyleGAN XL [30], Consistency Diffusion
Table 2: Detailed results on distinguishing belonging images and real images.
Model Type	Model	Training Dataset	Belongings vs Training Data	Belongings vs Unseen Data
TP	FP	FN	TN	Acc	TP	FP	FN	TN	Acc
Unconditional	DCGAN	CIFAR-10	96	0	4	100	98.0%	95	0	5	100	97.5%
VAE	CIFAR-10	95	0	5	100	97.5%	96	0	4	100	98.0%
Class-conditional	StyleGAN2-ADA	CIFAR-10	96	2	4	98	97.0%	95	0	5	100	97.5%
Consistency Model	ImageNet	96	24	4	76	86.0%	96	11	4	89	92.5%
Text-conditional	ControlGAN	CUB-200-2011	95	10	5	90	92.5%	96	8	4	92	94.0%
Table 3: Results for distinguishing belonging images and images generated by other models with different architectures. Here, Model  $\mathcal{M}_1$  is the examined model, Model  $\mathcal{M}_2$  is the other model that has same training data but different architectures.
Training Dataset	Model M1	Model M2	TP	FP	FN	TN	Acc
CIFAR-10	DCGAN	VAE	96	0	4	100	98.0%
StyleGAN2ADA	96	0	4	100	98.0%
DDIM	97	0	3	100	98.5%
DDPM	96	0	4	100	98.0%
TransGAN	97	0	3	100	98.5%
VAE	DCGAN	97	0	3	100	98.5%
StyleGAN2ADA	95	0	5	100	97.5%
DDIM	96	0	4	100	98.0%
DDPM	96	0	4	100	98.0%
TransGAN	97	0	3	100	98.5%
StyleGAN2ADA	DCGAN	96	2	4	98	97.0%
VAE	95	1	5	99	97.0%
DDIM	96	1	4	99	97.5%
DDPM	96	0	4	100	98.0%
TransGAN	95	6	5	94	94.5%
ImageNet	Consistency Model	StyleGAN XL	95	8	5	92	93.5%
StyleGAN XL	Consistency Model	96	10	4	90	93.0%
CUB-200-2011	ControlGAN	StackGAN-v2	96	17	4	83	89.5%
StackGAN-v2	ControlGAN	96	14	4	86	91.0%
Model [4], Control-GAN [28], StackGAN-v2 [29], and Stable Diffusion v2 [3]. These models are representative image generation models.
Performance Metrics. The effectiveness of the method is measured by collecting the detection accuracy (Acc). For a particular model, given a set of belonging images and non-belonging images, the Acc is the ratio between the correctly classified images and all images. We also show a detailed number of True Positives (TP, i.e., correctly detected belongings), False Positives (FP, i.e., non-belongings classified as belongings), False Negatives (FN, i.e., belongings classified as non-belongings) and True Negatives (TN, i.e., correctly classified non-belongings).
5.2 Effectiveness
In this section, we evaluate the effectiveness of RONAN from two perspectives: (1) its effectiveness in distinguishing between belongings of a particular model and real images; (2) its effectiveness in differentiating between belongings of a particular model and those generated by other models.
Distinguishing Belonging Images and Real Images. To investigate RONAN's effectiveness in distinguishing between belonging images of a particular model and real images, given an image generation model, we start by differentiating between the generated images of the given model and the training data of the model. The investigated models include DCGAN [1], VAE [6], StyleGAN2-ADA [10] trained on the CIFAR-10 [48] dataset, Consistency Model [4] trained on the ImageNet [53] dataset, and ControlGAN [28] trained on the CUB-200-2011 [57] dataset. Among them, DCGAN and VAE are unconditional image generation models. StyleGAN2-ADA and the latest diffusion model Consistency Model, are class-conditional models. In addition to distinguishing belongings and the training data, we also conduct experiments to distinguish belongings from unseen data that has a similar distribution to the training data (i.e., the test data of the dataset). For each case, we evaluate the results on 100 randomly sampled belonging images and 100 randomly sampled non-belonging images. The results are demonstrated in Table 2. As can be observed, the detection accuracies (Acc) are above  $85\%$  in all cases. On average, the Acc is  $94.2\%$  for distinguishing belongings and training data, and is  $95.9\%$  for distinguishing belongings and unseen data. The results demonstrate that
RONAN achieves good performance in distinguishing between belonging images of a particular model and real images.
Distinguishing Belonging Images and Images Generated by Other Models. In this section, we study RONAN's effectiveness to distinguish between belonging images of a particular model and the images generated by other models. For a given model  $\mathcal{M}_1$ , we consider two different types of other models  $\mathcal{M}_2$ , i.e., the model trained on the same dataset but with different architectures and the model that has the same architecture but is trained on a different dataset.
Models with Different Architectures. We first evaluate RONAN's effectiveness in distinguishing between belonging images of a particular model and the images generated by other models with the same training data but different architectures. For the models trained on the CIFAR-10 [48] dataset, the used model architectures are DCGAN [1], VAE [6], StyleGAN2-ADA [10], DDIM [2], DDPM [7], TransGAN [56]. For the Imagenet [53] dataset, the involved models are the latest diffusion model Consistency Model [4] and StyleGAN XL [30]. For the CUB-200-2011 [57] dataset, we use text-to-image models ControlGAN [28] and StackGAN-v2 [29]. To measure the effectiveness of RONAN, we collect its results on 100 randomly sampled belonging images and 100 randomly sampled non-belonging images. The results are shown in Table 3, where Model  $\mathcal{M}_1$  denotes the examined model, and Model  $\mathcal{M}_2$  represents the other model that has the same training data but a different architecture. The results show that the average detection accuracy (Acc) of RONAN is  $96.45\%$ , confirming its good performance for distinguishing between belongings of a given model and the images generated by other models with different architectures.
Models Trained on Different Datasets. We also evaluate RONAN's effectiveness in distinguishing between belonging images of a particular model and the images generated by other models with the same model architecture but trained on different datasets. The model used here is the diffusion model Consistency Model [4]. We use a model trained on the ImageNet [53] dataset and a model trained on the LSUN [58] dataset. The results are demonstrated in Table 4,
where Model  $\mathcal{M}_1$  denotes the examined model, and Model  $\mathcal{M}_2$  means the other model that has the same model architecture but is trained on a different dataset. The results show that RONAN can effectively distinguish between belonging images of a particular model and the images generated by other models with the same model architecture but different training data. On average, the detection accuracy of our method is  $93.0\%$ . In the Appendix, we also discuss the results when the training data of the
Table 4: Results for distinguishing belonging images and images generated by other models with different training data. Here, Model  $\mathcal{M}_1$  is the examined model, Model  $\mathcal{M}_2$  is the other model that has same architecture but different training data.
Trainging dataset of Model M1	Trainging dataset of Model M2	TP	FP	FN	TN	Acc
ImageNet	LSUN	95	13	5	87	91.0%
LSUN	ImageNet	96	6	4	94	95.0%
model  $\mathcal{M}_2$  and that of the model  $\mathcal{M}_1$  have overlaps. Empirical results demonstrate that RONAN is still effective even the training data of the examined model and that of the other model are similar (i.e., they have large overlaps).
5.3 Case Study on Stable Diffusion v2
In this section, we conduct a case study on the recent Stable Diffusion v2 [3] model. We first randomly collect 20 images of Shiba dogs from the internet and use these images as the nonbelonging images. We then use the prompt "A cute Shiba on the grass." and feed it into the Stable Diffusion v2 model to generate 20 belonging images. We apply RONAN on the model, and evaluate its performance in distinguishing the belonging images and non-belonging images. The results show that the detection accuracy of RONAN is  $87.5\%$ , with 18 TP, 3 FP, 2 FN, 17 TN. In Fig. 3, we show the visualization of a belonging image and a non-belonging image, as well as their corresponding reverse-engineered images. Note that the non-belonging image and the belonging image have similar inherent complexities (i.e., their reconstructed loss with the MSE metric on the reference model are 0.029 and 0.022, respectively). For the non-belonging image, the reverse-engineered image is more noisy and blurred, while the reverse-engineered image of the belonging image seems nearly identical to the original image. These results show the potential to apply our method on state-of-the-art models such as Stable Diffusion v2. More visualizations and examples can be found in Appendix J.
In this section, we evaluate the impact of different metrics used in calculating reconstruction loss, and the impact of reconstruction loss calibration.
Non-belonging
Reverse-engineered
Different Metrics. In the reverse-engineering sure the distance between different images. By In addition to MSE, we also evaluate the results on other image distance metrics, i.e., MAE (Mean Absolute Error) [59], SSIM (Structural Similarity Index Measure) [60], and LPIPS (Learned Perceptual Image Patch Similarity) [61]. The task is to distinguish between belonging images and real images (i.e., training images of the model here). The model used in this section is the StyleGAN2-ADA [10] trained on
task (Eq. 1), we use a metric  $\mathcal{L}$  to mea-default, we select MSE [49] as the metric.
Table 5: Results on different distance metrics.
Metric	TP	FP	FN	TN	Acc
MAE	96	7	4	93	94.5%
MSE	96	2	4	98	97.0%
SSIM	97	26	3	74	85.5%
LPIPS	96	4	4	96	96.0%
CIFAR-10 [48] dataset. The results are shown in Table 5. As we can observe, the detection accuracy (Acc) under MAE, MSE, SSIM, and LPIPS are  $94.5\%$ ,  $97.0\%$ ,  $85.5\%$ , and  $96.0\%$ , respectively. Overall, the MSE metric achieves the highest performance in distinguishing belonging images and real images. Thus, we select MSE as our default metric.
Impacts of Reconstruction Loss Calibration. To eliminate the influence of images' inherent complexities, we calibrate the reconstruction loss by considering the hardness
of the reverse-engineering on a reference model (§ 4.2). To measure the effects of the calibration step, we compare the detailed TP, FP, FN, TN, and Acc for the method with and without the calibration step. We use the Stable Diffusion v2 [3] model and follow the experiment settings
Table 6: Effects of reconstruction loss calibration.
Method	TP	FP	FN	TN	Acc
w/o Calibration	17	7	3	13	75.0%
w/ Calibration	18	3	2	17	87.5%
described in § 5.3. The results in Table 6 demonstrate the detection accuracy for the method with and without the calibration step are  $75.0\%$  and  $87.5\%$ , respectively. These results show that the calibration step can effectively improve the performance of RONAN.
5.5 Comparison to Existing Reconstruction based Attribution Methods
Albright et al. [43] and Zhang et al. [44] are two existing attribution methods that also apply the reconstruction on the input. In this section, we compare our method to them. Given an inspected image, Albright et al. and Zhang et al. works by enumerating and conducting inversion on all models within a set of suspicious candidate models (referred to as the "candidate set" in this section), and attribute the model with the lowest reconstruction loss as the source of the image. Their methods rely on the assumptions that the inspector can have the white-box access to all models in the candidate set, and the examined image must be generated by one of the models in the candidate set. These assumptions diminish the practicability of their methods, whereas our method does not have such requirements. Thus, our threat model and experiment settings are fundamentally different to theirs. Our paper focuses on the problem of determining if a given image is generated by a single given model or not. Consider a scenario where a model owner wants to verify if an image is generated by a model owned by him/her. While our method only needs to conduct the inversion on this specific model, Albright et al. and Zhang et al. need to compare the reconstruction loss of this particular model with a large number of suspicious models. There are several cases where Albright et al. and Zhang et al. are ineffective in addressing this problem: ① In cases where the inspector lacks white-box access
to some of the suspicious models, deriving reconstruction on them and getting inference results becomes infeasible. Notably, more and more state-of-the-art image generation models (e.g., such as Midjourney [62] and DALL-E2 [13]) are close-sourced and they only provide the black-box API to the users. ② Albright et al. and Zhang et al. are prone to making wrong predictions if the real source model is not included in the candidate set. This is attributed to their underlying assumption that the examined image must originate from one of the models within the candidate set. Ensuring the real source model is included within the candidate set is a very hard problem in practice. ③ Equally noteworthy, Albright et al. and Zhang et al. do not work when the inspected images are real images due to their strong assumption (i.e., the examined image must be generated by one of the model in the candidate set). Our method does not have the above problems.
Despite the threat models are different, we also conduct the comparison experiments. We consider the setting for distinguishing the belonging images of the inspected model  $\mathcal{M}_1$  and the generated images of other models  $\mathcal{M}_2$ , and the inspected model here is Stable Diffusion v2. For
Table 7: Comparison to Albright et al. [43] and Zhang et al. [44].
Model M1	Model M2	Method	TP	FP	FN	TN	Acc
Stable Diffusion v2	StyleGAN2-ADA	Albright et al.	100	100	0	0	50.0%
Zhang et al.	100	100	0	0	50.0%
Ours	96	7	4	93	94.5%
Consistency Model	Albright et al.	100	100	0	0	50.0%
Zhang et al.	100	100	0	0	50.0%
Ours	95	10	5	90	92.5%
our method, we assume the inspector can only access  $\mathcal{M}_1$ . For Albright et al. and Zhang et al., we assume the inspector can access both  $\mathcal{M}_1$  and  $\mathcal{M}_2$ . The results can be found in Table 7. While the average detection accuracy of Albright et al. and Zhang et al. are only  $50\%$ , our method achieves over  $90\%$  detection accuracy, meaning that our method outperforms Albright et al. and Zhang et al.
Laszkiewicz et al. [45] is a concurrent work focusing on the single-model attribution problem. Although the method proposed in Laszkiewicz et al. achieves promising performance, it is important to note that it is not model-agnostic. More specifically, it assumes the last layer of the inspected model is invertible. Therefore, it is not applicable to the models that use non-invertible activation functions (such as the commonly-used ReLU function) in the last layer. It also has assumptions on the architectures of the models, making it can not be used for many modern diffusion models [7, 63].
Limitations. While our method can achieve origin attribution in a alteration-free and model-agnostic manner, the computation cost might be higher than that of watermark-based methods [20-23] and classifier-based methods [24-27]. More discussion about the efficiency of our method can be found in the Appendix. Speeding up the reverse-engineering will be our future work. This paper focuses on the image generation models. Expanding our method for origin attribution on the model in other fields, (e.g., video generation models such asImagen Video [64], language generation models like ChatGPT [65], and graph generation models [66, 67]) will also be our future work.
Ethics. Research on security and privacy of machine learning potentially has ethical concerns [68-78]. This paper proposes a method to safeguard the intellectual property of image generation models and monitor their potential misuse. We believe that our approach will enhance the security of image generation models and be beneficial to generative AI.
In this paper, we take the first effort to introduce the "alteration-free and model-agnostic origin attribution" task for AI-generated images. Our contributions for accomplishing this task involves defining the reverse-engineering task for generative models and analyzing the disparities in reconstruction loss between generated samples of a given model and other images. Based on our analysis, we devise a novel method for this task by conducting input reverse-engineering and calculating the corresponding reconstruction loss. Experiments conducted on different generative models under various settings demonstrate that our method is effective in tracing the origin of AI-generated images.
We thank the anonymous reviewers for their valuable comments. This research is supported by Sony AI, IARPA TrojAI W911NF-19-S-0012, NSF 2342250 and 2319944. Any opinions, findings, and conclusions expressed in this paper are those of the authors only and do not necessarily reflect the views of any funding agencies.
[73] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
[74] Zhenting Wang, Hailun Ding, Juan Zhai, and Shiqing Ma. Training with more confidence: Mitigating injected and natural backdoors during training. Advances in Neural Information Processing Systems, 35:36396-36410, 2022.
[75] Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. Query-efficient hard-label black-box attack: An optimization-based approach. arXiv preprint arXiv:1807.04457, 2018.
[76] Zhenting Wang, Kai Mei, Hailun Ding, Juan Zhai, and Shiqing Ma. Rethinking the reverse-engineering of trojan triggers. Advances in Neural Information Processing Systems, 2022.
[77] Zhenting Wang, Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15074-15084, 2022.
[78] Kai Mei, Zheng Li, Zhenting Wang, Yang Zhang, and Shiqing Ma. Notable: Transferable backdoor attacks against prompt-based nlp models. Annual Meeting of the Association for Computational Linguistics, 2023.
[79] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010.
[80] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695-8704, 2020.
[81] Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[82] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
[83] Robert B Ash. Information theory. Courier Corporation, 2012.
[84] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740, 2017.
Roadmap: In this appendix, we show the proof for Theorem 4.2 (Appendix A), detailed process for calculating critical value of the  $t$  distribution used in  $\S 4.3$  (Appendix B), more results for distinguishing belonging images and images generated by other models (Appendix C), a thought experiment on VAE (Appendix D), results on different reverse-engineering methods (Appendix E), discussion on different methods for measuring the complexity of the images (Appendix F). RONAN's robustness to editing-based adaptive attack (Appendix G), the discussion for the efficiency of RONAN (Appendix H). Finally, we provide more results and demonstrate more visualizations on Stable Diffusion v2 [3] (Appendix I and Appendix J).
A Proof for Theorem 4.2
We start our analysis from ideal generative model and reconstruction algorithm, which we define as deterministic generative model and perfect reconstruction algorithm:
Definition A.1 (Deterministic Generative Model). Given a generative model  $\mathcal{M}:\mathcal{I}\mapsto \mathcal{X}_{\mathcal{M}}$ , it is deterministic if it always produces the same output  $\pmb {x}\in \mathcal{X}_{\mathcal{M}}$  given the same input  $\pmb {i}\in \mathcal{I}$
Definition A.2 (Perfect Reverse-engineering Algorithm). Given a reverse-engineering algorithm  $\mathcal{A}$ , if it is guaranteed that the returned reconstruction loss  $l$  is the global minima, then we say  $\mathcal{A}$  is a perfect reverse-engineering algorithm.
Proof. Assume the given output sample  $\pmb{x}$  is generated by input  $\pmb{i}$ . Since the given model  $\mathcal{M}$  is deterministic, we have:
$$
\boldsymbol {x} = \mathcal {M} (\boldsymbol {i}) \tag {4}
$$
In this case, the distance between the  $\mathbf{x}$  and  $\mathcal{M}(i)$  is 0, i.e.,  $\mathcal{L}(\mathcal{M}(i),\mathbf{x}) = 0$ . Based on Theorem 4.1, as  $\mathcal{A}$  is a perfect reverse-engineering algorithm, it can find the input that can produce the minimal reconstruction loss. Therefore, we have:
$$
\forall \boldsymbol {x} \in \mathcal {X} _ {\mathcal {M}}, \mathcal {A} (\mathcal {M}, \boldsymbol {x}) = 0 \tag {5}
$$
Similarly, for sample  $\pmb{x}' \notin \mathcal{X}_{\mathcal{M}}$ . There does not exist an input  $i'$  that can produce  $\pmb{x}'$ , meaning that there does not exist an input  $i'$  that have  $\mathcal{L}(\mathcal{M}(i'), \pmb{x}') = 0$ . Thus, we have:
$$
\forall \boldsymbol {x} ^ {\prime} \notin \mathcal {X} _ {\mathcal {M}}, \mathcal {A} (\mathcal {M}, \boldsymbol {x} ^ {\prime}) > 0 \tag {6}
$$
Finally, we have for any  $\pmb{x} \in \mathcal{X}_{\mathcal{M}}$  and  $\pmb{x}' \notin \mathcal{X}_{\mathcal{M}}$  we have  $\mathcal{A}(\mathcal{M}, \pmb{x}') > \mathcal{A}(\mathcal{M}, \pmb{x})$ .
B Computing Critical Value of the  $t$  Distribution.
In § 4.3, we use the critical value of the  $t$  distribution to obtain the results of the hypothesis testing. In this section, we discuss the detailed process for calculating the critical value. For the t-distribution, we have the probability density function: