Responsible Disclosure of Generative Models Using Scalable Fingerprinting
Authors: Ning Yu
Vladislav Skripniuk
Dingfan Chen
Larry S. Davis
Mario Fritz
Venue: ICLR 2022 Spotlight (2022)
Keywords: Generative models
fingerprinting
responsible disclosure
deep fake detection and attribution

Abstract: Over the past years, deep generative models have achieved a new level of performance. Generated data has become difficult, if not impossible, to be distinguished from real data. While there are plenty of use cases that benefit from this technology, there are also strong concerns on how this new technology can be misused to generate deep fakes and enable misinformation at scale. Unfortunately, current deep fake detection methods are not sustainable, as the gap between real and fake continues to close. In contrast, our work enables a responsible disclosure of such state-of-the-art generative models, that allows model inventors to fingerprint their models, so that the generated samples containing a fingerprint can be accurately detected and attributed to a source. Our technique achieves this by an efficient and scalable ad-hoc generation of a large population of models with distinct fingerprints. Our recommended operation point uses a 128-bit fingerprint which in principle results in more than 10^{38} identifiable models. Experiments show that our method fulfills key properties of a fingerprinting mechanism and achieves effectiveness in deep fake detection and attribution. Code and models are available at https://github.com/ningyu1991/ScalableGANFingerprints.

Published as a conference paper at ICLR 2022
RESPONSIBLE DISCLOSURE OF GENERATIVE MODELS USING SCALABLE FINGERPRINTING
Ning Yu 12.3 Vladislav Skripniuk*1 Dingfan Chen
Larry Davis Mario Fritz*
"Sllssforee Research Universityy of Maryland
"Max Planck Institute for Informatics
*CISPA Helmholtz Center for Information Security
ning. wwwsalesforce.e.ooooooo com Aladislavémpe cemfpp-inf... .in.iin...........mm.....m...... de {dingfan. chen, fritz)@cispa.dde lsdêcs , umd. edu
ABSTRACT
Over the past years, deep generative models have achieved a new level of perfor- mance. Generated data has become difficult, if not impossible, to distinguished from real data. While there are plenty of use cases that benefit from this technology, there are also strong concerns on how this new technology can be misused to generate deep fakes and enable misinformation scale. Unfortunately, current deep fake detection methods are not sustainable, as the gap between real and fake continues close. In contrast, our work enables responsible disclosure of such state-of-the-art generative models, that allows model inventors to fingerprint their models, so that the generated samples containing a fingerprint can be accurately detected and attributed to a source. Our technique achieves this by an efficient and scalable ad-hoc generation of large population of models with distinct fingerprints. Our recommended operation point uses a 128-bit fingerprint which in principle results more than 1008 identifiable models. Experiments show that our method fulfills key properties of fingerprinting mechanism and achieves effectivvness in deep fake detection and attribution. Code and models are available at GitHub.
INTRODUCTION
Over the recent seven years, deep generative models have demonstrated stunning performance in generating photorealistic images, considerably boosted by the revolutionary technique of generative adversarial networks (GANs) (Goodfellow et al., 2014; Radford et al., 2016; Gulrajani et al., 2017; Miyato et al., 2018; Brock et al., 2018; Karras et al., 2018; 2019; 2020).
However, with the closing margins between real and fake, a flood of strong concerns arise (Harris, 2018; Chesney & Citron, 2019; Brundage al., 2018): how if these models misused to spoof sensors, generate deep fakes, and enable misinformation at scale? Not only human beings have difficulties in distinguishing deep fakes, but dedicated research efforts deep fake detection (Durall et al., 2019; Zhang et al., 2019; Frank et al., 2020; Zhang et al., 2020) and attribution (Marra et al., 2019; Yu al., 2019; Wang et al., 2020) are also unable to sustain longer against the evolution of generative models. For example, researchers delve into details on how deep fake detection works, and learn to improve generation that better fits the detection criteria (Zhang et al., 2020; Durall et al., 2020). In principle, any successful detector can play an auxiliary role in augmenting the discriminator in the next iteration of GAN techniques, and consequently results in an even stronger generator.
The dark side of deep generative models delays its industrialization process. For example, when commercializing the GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020) models, OpenAl leans conservative open-source their models but rather only release the black-box APIs They involve expensive human labor in the loop to review user downloads and monitor the usage of their APIs. Yet still, it is challenging and industry-wide task how to trace the responsibility of the downstream use cases in an open end.
To pioneer in this task, propose model fingerprinting mechanism that enables responsible release and regulation of generative models. In particular, we allow responsible model inventors to
"This work was done when Ning Yu was in joint Ph.D. program with the University of Maryland and Max Planck Institute for Informaties.
Equal contribution.
https:: pttpss:/ccen //tpeaaii.i..//////////eo com/ foom/blow http://logoopeaiiiigg -api/

---

Published as a conference paper at ICLR 2022
fingerprint their generators and disclose their responsibilities. As result, the generated samples contain fingerprints that can be accurately detected and attributed to their sources. This is achieved by an efficient and scalable ad-hoc generation of a large population of generator instances with distinct fingerprints. See Figure Middle.
Similar in the spirit of the dynamic filter networks (Jia et al., 2016) and style-based generator architec- tures (Karras et al., 2019; 2020) where their network filters are not freely learned but conditioned on an input, we learn to parameterize unique fin- gerprint into the filters of each genera- tor instance. The core gist is to incor- porate a fingerprint auto-encoder into a GAN framework while preserving the original generation performance. See Figure Left. In particular, given GAN backbone, we the fingerprint embedding from the encoder modulate each convolutional filter of the generator (Figure 2(b)), and try to decode this fingerprint from the generated images. We jointly train the fingerprint auto-encoder and GAN with our fingerprint related losses and the original adversarial loss. See Figure 2(a) for the diagram, and for details.
Training
Deployment sperlpp Fingarpeira
Detection
User-spacffc Regerprinted Generaters
Random Ringerpents cc (I
G
G3
Fingerprit
Reconstructin
Figure I:
The diagram of our fingerprinting mechanism for generators. See main text for descriptions.
After training, the responsible model inventor is capable of efficiently fingerprinting and releasing different generator instances to different user downloads, which are equipped with the same generation performance but with different fingerprints. Each user download corresponds to unique fingerprint, which is maintained by the inventor's 's database. As result, when misuse of a model happens, the model inventor can use the decoder to detect the fingerprint from the generated images, match it in the database, and then trace the responsibility of the user. See Figure Right. Based on this form of responsible disclosure, responsible model inventors, like OpenAl, have a way to mitigate adverse side effects society when releasing their powerful models, while at the same time should have an automatic way to attribute misuses.
There are several key properties of mechanism. The efficiency to instantiate a generator is inherently satisfied because, after training, the fingerprint encoding and filter modulation run with little overhead. We evaluate the effectiveness of our fingerprinting and obtain almost perfect fingerprint detection accuracy. We also justify the fidelity with a negligible side effect on the original generation quality. See Section 4. Our recommended operation point uses 1 128-bit fingerprint (Section 4.2) which in principle results in a capacity of more than 10a8 identifiable generator instances. The scalability benefits from the fact that fingerprints are randomly sampled on the fly during training so that fingerprint detection generalizes well for the entire fingerprint space. See Section 4.3 for validation. In addition, we validate in Section 4.4 the secrecy of presence and value of our fingerprints against shadow model attacks. We validate in Section 4.5 the robustness and immunizability against perturbation on generated images.
To target the initial motivation, we move the deep fake detection and attribution solutions from passive detectors to proactive fingerprinting. We show in Section 4.6 saturated performance and advantages over two ofte-offthe-art discriminative methods (Yu et al., 2019; Wang et al., 2020) especially in the open world. This is because, conditioned on user-specific fingerprint inputs, the presence of such fingerprints in generated images guarantees the margin between real and fake, and facilitates the attribution and responsibility tracing of deep fakes to their sources.
Our contributions are in four thrusts: (1) We enhance the concept of fingerprinting for generative models that enables a responsible disclosure of state-of-the-art GAN models. (2) We pioneer in the novel direction for efficient and scalable GAN fingerprinting mechanism, i.e., only one generic GAN model is trained while more than 1098 fingerprinted generator instances can be obtained with little overhead during deployment. (3) We also justify several key properties of our fingerprinting, including effectiveness, fidelity, large capacity, scalability, secrecy, robustness, and immunizability. (4) Finally, for the deep fake detection and attribution tasks, we move the solution from passive classifiers to proactive fingerprinting, and validate its saturated performance and advantages. It makes our responsible disclosure independent of the evolution of GAN techniques.
2

---

Published as a conference paper at ICLR 2022
2
RELATED WORK
Deep fake detection and attribution. These tasks come along with the increasing concerns on deep fake misuse (Harris, 2018: Chesney & Citron, 2019; Brundage et al., 2018). Deep fake detection is binary classification problem to distinguish fake samples from real ones, while attribution further traces their sources. The findings of visually imperceptible but muucinn-dissinguishabbl patterns in GAN-generated images make these tasks viable by noise pattern matching (Marra et al., 2019), deep classifiers (Afchar et al., 2018; Hsu et al., 2018; Yu et al., 2019), or deep Recurrent Neural Networks (Güera & Delp, 2018). (Zhang et al., 2019; Durall et al., 2019; 2020; Liu et al., 2020) observe that mismatches between real and fake in frequency domain or in texture representation can facilitate deep fake detection. (Wang et al., 2020; Girish et al., 2021) follow up with generalization across different GAN techniques towards open world. Beyond attribution, (Albright & McCloskey, 2019; Asnani et al., 2021) even reverse the engineering to predict in the hyper-pen parameerr space of potential generator sources.
However, these passive detection methods heavily rely on the inherent clues in deep fakes. Therefore, they can barely sustain a long time against the adversarial iterations of GAN techniques. For example, (Durall et al., 2020) improves generation realism by closing the gap in generated high-frequency components. To handle this situation, artificial fingerprinting is proposed in (Yu et al.. 2021) to proactively embed clues into generative models by rooting fingerprints into training data. This makes deep fake detection independent of GAN evolution. Yet, as indirect fingerprinting, (Yu et al., 2021) cannot scale up large number of fingerprints because they have to pre-process training data for each individual fingerprint and re-train generator with each fingerprint. Our method is similar in spirit to (Yu et al., 2021), but possesses fundamental advantages by directly and efficiently fingerprinting generative models: after training one generic fingerprinting model, can instantiate large number of generators ad-hoc with different fingerprints.
Image steganography and watermarking, Steganography targets to manipulate carrier images in hidden manner such that the communication through the images can only be understood by the sender and the intended recipient (Fridrich, 2009). Traditional methods rely on Fourier transform (Cox et al., 2002; Cayre et al., 2005), JPEG compression". 23 or least significant bits modification (Pevný et al., 2010; Holub et al., 2014). Recent works utilize deep neural encoder and decoder hide information (Baluja, 2017; Tancik et al., 2020; Luo et al., 2020). Watermarking targets to embed ownership information into carrier images such that the owner's identity and authenticity can be verified. It belongs to a form of steganography that sometimes interacts with physical images (Tancii et al., 2020). Existing methods rely on log-polar frequency domain (Pereira & Pun, 2000; Kang et al., 2010). printer-camert transform (Solanki et al., 2006; Pramila et al., 2018), or display-camera transform (Yuan et al., 2013; Fang et al., 2018). Recent works also use deep neural networks to detect when an image has been re-imaged (Fan et al., 2018; Tancik et al., 2020). The concept and function of our fingerprinting solution similar in spirit of watermarking, but differs fundamentally. In particular, we did not retouch individual images. Rather, our solution is the first retouch generator parameters so as to encode information into the model.
Network watermarking, Network watermarking techniques (Uchida et al., 2017; Adi et al., 2018; Zhang et al., 2018; Chen et al., 2019; Rouhani et al., 2019; Ong et al., 2021; Yu et al., 2021) embed watermarks into network parameters rather than pixels while not deteriorating the original utility. Our solution shares motivations with them but substantially differs in terms of concepts, motivations, and techniques. For concepts, most existing works applicable to only image classification models, only (Ong et al., 2021: Yu et al., 2021) work for generative models but suffer from poor efficiency and scalability, For motivations, the existing works target to fingerprint a single model, while we are motivated by the limitation of (Ong et al., 2021; Yu et al., 2021) to scale up the fingerprinting to as many as 10 various generator instances within one-time training. For techniques, most existing works embed fingerprints in the input-output behaviors (Adi et al., 2018; Zhang et al., 2018; Ong et al., 2021), I), while our solution gets rid of such trigger input for improved scalability.
httt::://ww..uttuuusssss.
hhttp://steghie de http:://ttaghdeee..ooorrrrr.r../
3

---

Published as a conference paper at ICLR 2022
Fingerprint (ex(0
Latent Code A00, )
Mocidaled Conv
Cude
Fingerprin
Upcampp
Poleen
Mocilatted
Co
Comy
Channel-wit Modulate
Fingarprint
Fingerprinted
Convolutionll Flter
Fingorpritted Filerr W c gelde
Embeddiing
G(s.c))
Images
G(s.c))
(a) Pipeline.
(b) Modulated convolutional layer.
Figure 2: The diagrams of our fingerprinting pipeline and the modulated convolutional layer.
3 GAN FINGERPRINTING NETWORKS
Throughout the paper, we stand for the responsible model inventor's perspective, which is regarded as the regulation hub of our experiments. None of the encoder, decoder, and training data should be accessible to the public. Only fingerprinted generator instances are released to the open end.
We list symbol notations at the beginning. We use latent code No Id:) to control generated contents. We set d, = 512. We represent fingerprint с ~ Ber(0.5)% as a sequence bits. follows Bernoulli distribution with probability 0.5. We non-trivially choose the fingerprint length de in Section 4.2. We denote encoder E mapping c to its embedding, generator G mapping (z, E(c)) to the image domain, discriminator D mapping probability, and decoder F mapping an image to the decoded latent code and fingerprint (ž, C). In the following formulations, denote G(z, E(c)) G(z, c) for brevity.
image
Pduta to the real/fake classification
3.1 PIPELINE
We consider three goals in our training. First, we preserve the original functionality of GANs to generate realistic images, close to real distribution as possible. We use the unsaturated logistic loss as in (Goodfellow al., 2014; Karras et al., 2019; 2020) for real/fake binary classification:
log (1 D(G(2,)))
Ladd
E log ((x) +
c~Beer00.5)
In addition, similar (Srivastava al., 2017), reconstruct the latent code through the decoder F to augment generation diversity and mitigate the mode collapse issue of GANs (Srivastava et al., 2017: Li Malik, 2018; Yu al., 2020).
C = 1
Zk F(G(2,))),)
(2)
1=1
c~
where we use the first d, output elements of F that correspond the decoded latent code. The second goal is to reconstruct the fingerprint so as to allow fingerprint detection.
de
C.
E
ck log (F((G(z.c) (1-ca) (1-
(I=O((GG(..))a (3)
2 (D.5)
where the last de output elements of F as the decoded fingerprint. o(-) denotes the sigmoid function that diffcrentiably clips the output to the range of [0, 1]. The reconstruction is therefore a combination of cross-entropy binary classification for each bit.
worth noting that we use one decoder to decode both the latent code and fingerprint, which benefits explicit disentanglement between their representations discussed below.
The third goal is to disentangle the representation between latent code and fingerprint. Desirably, latent code should have exclusive control over the generated content. This sticks to the original
4

---

Published as a conference paper at ICLR 2022
generation functionality. Therefore, two images with different fingerprints but with identical latent code should have a consistent appearance. We formulate the consistency loss as:
Comst
E
(I)(z, G(z, c2)ll
The disentangled effect demonstrated in Figure and Appendix Figure
Our final training objective is as follows. We optimize it under the adversarial training framework w.r.t E, G. F. and D.
min max \ Lade + X2 + Azh. + AcCn P.G.
(5)
where A 1.0, A2 = 1.0, A3 = 2.0, and \4 = 2.0 are hyper-parameters to balance the magnitude of each loss term, Each loss contributes to property of our solution. The weight settings are empirically not sensitive within its magnitude level. See Figure 2(a) for the diagram.
3.2 FINGERPRINT MODULATION
At the architectural level, it is non-trivial how to embed E(c) into G. The gist is to embed fingerprints into the generator parameters rather than generator input, so that after training a generic model we instantiate a large population of generators with different fingerprints. This is critical to make fingerprinting efficient and scalable, as validated in Section 4.3. We then deploy only the fingerprinted generator instances to user downloads, not including the encoder.
We achieve this by modulating convolutional filters in the generator backbone with our fingerprint embedding, similar in spirit of (Karras et al., 2020). Given convolutional kernel W E R3x3xd; layer we first project the fingerprint embedding E(e) through an affine transformation Φi such that di (E(()) Rd: The transformation is implemented as a fully-connect neural layer with learnable parameters. We then scale each channel of W with the corresponding value in Φ1. In specific,
W ij.k (E(C)) Wij.A. Vi.j,k
See Figure 2(b) for diagram illustration. We compare to the other fingerprint embedding archi- tectures in Section 4. I and validate the advantages of this one. We conduct modulation for all the convolutional filters at layer with the same fingerprint embedding. And we investigate in Appendix Section A.2 at which layer to modulate we can achieve the optimal performance. desirable trade-off is to modulate all convolutional layers.
Note that, during training, latent code and fingerprint e: are jointly sampled. Yet for deployment, the model inventor first samples fingerprint then modulates the generator G with Co, and then deploys only the modulated generator G(., Co) to a user. For that user there allows only one input, i.e. the latent code, to the modulated generator. Once misuse happens, the inventor uses the decoder to decode the fingerprint and attribute to the user, so as to achieve responsible disclosure.
4 EXPERIMENTS
Datasets. We conduct experiments CelebA face dataset (Liu al., 2015), LSUN Bedroom and Cat datasets (Yu et al., 2015). LSUN Cat is the most challenging one reported in StyleGAN2 (Karras al., 2020). We train/evaluate on 30k/30k CelebA, 30k/30k LSUN Bedroom at the size of 128 x 128, and 50k/50k LSUN Cat at the size of 256x256.
GAN backbone. We build upon the most recent state-of-the-arr StyleGAN2 (Karras et al., 2020) config E. This aligns to the settings in (Yu et al., 2021) and facilitates our direct comparisons. See Appendix for the implementation details.
4.1 EFFECTIVENESS AND FIDELITY
Evaluation. The effectiveness indicates that the input fingerprints consistently appear in the generated images and can be accurately detected by the decoder. This is measured by fingerprint detection bitwise accuracy over 30k random samples (with random latent codes and random fingerprint codes). We use 128 bits to represent a fingerprint. This a non-trivial setting as analyzed Section 4.2.
In addition, bit matching may happen by chance. Following (Yu et al., 2021), perform null hypothesis test to evaluate the chance, the lower the more desirable. Given the number of matching bits & between the decoded fingerprint and its encoded ground truth, the null hypothesis Ho is getting
5

---

Published as a conference paper at ICLR 2022
CelebA
LSUN Bedroom
LSUN Cat
Method
Bit acc
p-value
FID
Bit acc ft
p-value
FID 4
Bit ft
p-value 4
FID 4
StyleGiAN2
9.37
19.24
31.01
outguess
0.533
0.268
10.02
0.526
0.329
20.15
0.523
0.329
32.30
steghide
0.535
0,268
9.48
0.530
0.268
19.77
0.541
0.213
31.67
(Yu et al., 1)21)
0.989
< 10-36
14.13
0.983
10-34
21.31
0.990
10-36
32.60
Ours
0,991
< 10 36
11.50
0,993
36
20.50
0.996
36
33.94
Ours Variant I
0,999
< 10
12.98
0.999
10 as
20.68
0,500
0.535
34.23
Ours Variant II
0,987
< 10
13.86
0.927
< 10 26
21.70
0.869
< 10-
34.33
Ours Variunt III
0,990
< 10 18
22.59
0.896
< 10
64.91
0,901
10-23
51.74
Table 1: Fingerprint detection in bitwise accuracy with p-value to accept the null hypothesis test, and generation fidelity in FID. t/U indicates a higher/lower value more desirable. The baseline results are directly copied from (Yu et al., 2021).
this number of matching bits by chance. It is calculated as Pr(X > k Ho) = -ie (de) 0.54,, according to the binomial probability distribution with de trials, where de the fingerprint bit length. p-value should be lower than 0.05 to reject the null hypothesis.
The fidelity reflects how imperceptibly the original generation is affected by fingerprinting. It also helps avoid one's he's suspect of the presence of fingerprints which may attract adversarial fingerprint removal. We report Fréchet Inception Distance (FID) (Heusel et al., 2017) between 30k generated images and 30k real testing images. A lower value indicates the generated images are more realistic.
Baselines. We compare seven baseline methods. The first baseline the StyleGAN2 (Karras et al., 2020) backbone. It provides the upper bound of fidelity while has no fingerprinting functionality.
The second baseline Yu et al., 2021) which is the other proactive but indirecs fingerprinting method for GANs. Another two baselines, outguess' and steghide", are similar to (Yu et al., 2021). They just replace the deep image fingerprinting auto-encoder in (Yu et al., 2021) with traditional JPEG- compression-based image watermarking techniques, and still suffer from low efficiency/scalabiiityy
We also compare our mechanism to three architectural variants. The motivation of these variants is to incorporate fingerprints in different manners. Variant I: modulating convolutional filters with only latent code embedding, while instead feeding the fingerprint code through the input of the generator. This is to test the necessity of fingerprint modulation. Variant II: modulating filters twice, with latent code embedding and fingerprint code embedding separately. Variant III: modulating filters with the embedding from the concatenation of latent code and fingerprint code.
Results. From Table 1, we find that:
(1) The traditional image watermarking methods, outguess and steghide, fail to deliver fingerprints in generated images, indicated by the random guess (~ 0.5) detection accuracy. We attribute this to the representation gap between deep generative models and shallow watermarking techniques.
(2) On CelebA, all the other methods achieve almost perfect fingerprint detection accuracy with p-value close to zero. This is because CelebA is landmark-aligned dataset with limited diversity. Fingerprinting synergizes well with generation regardless of model configuration.
(3) On LSUN Bedroom and Cat, only (Yu al., 2021) and our optimal model obtain saturated fingerprint detection accuracy. Ours Variant I, II. and III do not always achieve saturated performance. Especially Ours Variant I fails on LSUN Cat. We reason that filter modulation is strong formulation for reconstruction. Modulating fingerprints is necessary for their detection while modulating latent code along with fingerprint code distracts fingerprint reconstruction.
(4) Our method has comparable performance to (Yu et al., 2021), plus substantial advantages in practice: during deptoyment, we can fingerprint a generator instance seconds, contrast to (Yu et al., 2021) that has to retrain a generator instance in 3-5 days. This is 50000 gain of efficiency.
(5) Our method results in negligible $2.93 FID degradation. This is worthy trade-off to introduce the fingerprinting function.
http::///wwwwotgguusss.o. http::/sttegiie www..eeeee..................

---

Published as a conference paper at ICLR 2022
(6) We show in Figure 3 and Appendix Figure 6 uncurated generated samples from several generator instances. Image qualities are high. Fingerprints imperceptible. Thanks to the consistency loss Leons in Eq. 4, different generator instances can generate identical images given the same latent code. Their fingerprints are clued only in the non-salient background and distinguishable by our decoder.
rapegee mappiie rugupae selly Sonny
4.2 CAPACITY
The capacity indicates the number of unique fingerprints mech- anism can accommodate without crosstalk between two fingerprints. This is determined by de, fingerprint bit length, and by our detection accuracy (according to Section 4.1). The choice of fingerprint bit length is however non-trivial. A longer length can accommodate more fingerprints but is more challenging to reconstruct/detect.
Figure 3: Fidelity and dis- entangled control on CelebA: generated samples from five generator instances. For each row, we use a unique finger- print to instantiate generator. For each column, we feed in the same latent code to the gen- erator instances. More sam- ples are in Appendix Figure 6.
To figure out the optimal fingerprint bit length, we conduct the following experiments. On one hand, given one length, we evaluate our detection accuracy. On the other hand, we estimate the bottom- line requirement for detection accuracy. This is simulated as the maximal percentage of bit overlap among large bag (1 million) of fingerprint samples. The gap between the detection accuracy and bottom-line requirement should be the larger the better.
In Figure 4, we vary the fingerprint bit length in the options of {32, 64, 128, 256, 512), and plot the bitwise detection accuracy in red and the bottom line requirement in blue. We find:
1.0
Bitwise accuracy
Testing
0.8
(I) The bottom line requirement is monotonically decreasing w.r.t. the bit length of fingerprint because larger bit length leads to less heavy fingerprint overlaps.
0.7
0.6
(2) The testing accuracy is also monotonically decreasing w.r.t. the bit length of fingerprints. This is due to the challenge of fingerprint reconstruction/deeectio.
100
200
300 400
500
bits
Figure 4: Capacity: finger- print detection bitwise accu- racy and its bottom line re- quirement w.r.t. fingerprint bit length CelebA.
(3) The testing accuracy is empirically decreasing more slowly at the beginning and then faster than its bottom-ling requirement. We, therefore, pick the bit length 128 as the optimal choice for the maximal gap. We stick to this for all our experiments.
(4) Considering our detection bitwise accuracy 20.991 and our fingerprint bit length as 128, we derive in principle our mechanism can hold a large capacity of 2 1228x0.901 1038 identifiable fingerprints.
4.3 SCALABILITY
Scalability is one of the advantageous properties of mechanism: during training, we can efficiently instantiate a large capacity of generators with arbi-
Fingerprint set size
Training acc
Testing acc it
10
1,000
0.512
trary fingerprints on the fly, so that fingerprint detec-
100
1.000
0.537
1.000
0.752
tion generalizes well during testing. To validate this,
10k
0.990
0,988
we compare the baselines where we intentionally downgrade our method with access to only a finite set of fingerprints. These baselines stand for the cate- gory of non-scalable fingerprinting methods that have to re-train a generator instance for each fingerprint, e.g. (Yu et al., 2021). We cannot directly compare to (Yu et al., 2021) because is impractical cume-consuming) to instantiate large number of their generators for analysis. As workaround, we trained our detector using < 1 real samples to simulate the nca-scalable nature of the baseline.
100k
0.983
0.981
Sampling on the fly
0.991
0.991
Table 2: Scalability: fingerprint detection in
bitwise accuracy w.r.t. set size CelebA. indicates a higher value is more desirable.
From Table 2 we show that fingerprint detection fails to generalize unless can instantiate generators with 10k or more fingerprint samples. This indicates the necessity to equip GANs with an efficient and scalable fingerprinting mechanism, preferably the one the fly.
7

---

Published as a conference paper at ICLR 2022
4.4 SECRECY
The presence and value of fingerprint should not be easily spotted by third party, otherwise it would be potentially removed. In fact, secrecy of our fingerprints is another advantageous property, because our fingerprint encoder, different from image steganography directly retouch generated images. As a result, traditional secrecy attack protocols, e.g. Artificial Training Sets (ATS) attack used in (Lerch-Hostalot & Megías, 2016; Yu et al., 2021), is not applicable.
watermarking, does not
Instead, we employ the shadow-model-basee attack (Salem et al., 2020) to try detecting the presence and value of a fingerprint from generated images. We assume the attacker can access the model inventor's 's training data, fingerprint space, and training mechanism. He re-trains his own shadow fingerprint auto-encoder. For the fingerprint presence attack, on CelebA dataset, the attacker trains ResNet-1 1 B--ased (He et al., 2016) binary classifier to distinguish 10k non-finged con-fingrpprinted images (5k real plus 5k generated) against 10k generated images from his fingerprinted generators. We find near-saturated 0.981 training accuracy. Then he applies the classifier to IK inventor's 's generated images. As a result, we find only 0.505 testing accuracy on the presence of fingerprints, close to random guess. For the fingerprint value attack, on CelebA dataset, the attacker applies his shadow decoder (0.991 training bitwise accuracy) to Ik inventor's 's generated images. As a result, we find only 0.513 testing bitwise accuracy, also close to random guess. We conclude that the mismatch between different versions of fingerprinting systems disables the attacks, which guarantees its secrecy.
4.5 ROBUSTNESS AND IMMUNIZABILITY
Deep fakes in the open end may undergo post- pest-processing environments and result quality deteri- oration. Therefore, robustness against image perturbations is equally important to our mechanism. When it does not hold for some perturbations, our immunizability property compensates for it.
Following the protocol in (Yu et al., 2019), we evaluate the robustness against five types of image perturbation: cropping and resizing, blurring with Gaussian kernel, JPEG compression, additive Gaussian noise, and random combination of them. We consider two versions of our model: the original version and the immunized version. An immunized model indicates that during training we augment generated images with the corresponding perturbation in random strengths before feeding them to the fingerprint decoder.
It is worth noting that none of the encoder, decoder, and training data are accessible to the public. Therefore, the robustness against perturbation has to be experimented with the black-box assumption, as protocoled in (Yu et al., 2019). In other words, white-box perturbations such adversarial image modifications (Goodfellow et al., 2015) and fingerprint overwriting, which requires access to the encoder, decoder, and/or training data, not applicable in our scenario.
We plot in Figure 5 the comparisons of fingerprint detection accuracy among our original//mmunized models and the models of (Yu et al., 2021) w.r.t. the strength of each perturbation. We find:
(I) For all the perturbations, fingerprint detection accuracy drops monotonically as we increase the strength of perturbation. For some perturbations in red plots, i.e., blurring and JPEG compression, accuracy drops slowly in a reasonably large range. We consider accepting accuracy 275%. As a result, the robust working range under blurring is: Gaussian blur kernel size [0, 7]: under JPEG compression is: JPEG quality ~ [80, 100). Usually, the images turn not functional with perturbations heavier than this range. We, therefore, validate the robustness against blurring and JPEG compression.
(2) For the other perturbations, although our original model is not robust enough, perturbed augmen- tation compensates significantly in blue dots. We consider accepting accuracy 275%. As a result, the immunized working range under cropping is: cropping size [60, 128]; under Gaussian noise
Cembined
Origial
Immurized
Steign bool
(a)
(b)
(C)
(d)
(e)
Figure 5: Robustness and immunizability: red/blue plots show, on CelebA, the fingerprint detection of our original/immunized model in bitwise accuracy w.r.t. the strength of perturbations. Green plots show those of (Yu et al., 2021) as references. The data points are directly copied from their paper.
8

---

Published as a conference paper at ICLR 2022
is: noise standard deviation ~ [0.0, 0.4]; under combined perturbation is: the combination of the original or immunized working ranges aforementioned.. We, therefore, validate the immunizability of our model against cropping, Gaussian noise, and the combined perturbation.
(3) Comparing between (Yu et al., 2021) and our models, for blurring, their model in green plot is less robust than our original/immunized models. For the other perturbations, theirs are more robust than our original models but are outperformed by our immunized models. This indicates the importance of immunizability of fingerprinting solution, which is however lacking in (Yu et al., 2021).
4.6 DEEP FAKE DETECTION AND ATTRIBUTION
The effectiveness, robustness, and immunizability in turn benefit our initial motivation: deep fake detection and attribution. The former task is a binary classification problem to distinguish between real and fake. The latter task is to further finely label the source of a generated image.
We move the solution from passive classifiers to proactive fingerprinting, and merge the two tasks into one with I+N classes: 1 real-worth source and N GAN sources, where N can be extremely large, as large as our capacity 10 in Section 4.2. Then the tasks are converted to verifying if one decoded fingerprint is in our database or not. This achieved by comparing the decoded fingerprint to each fingerprint in the database given a threshold of bit overlap. According to our 200991 fingerprint detection accuracy, it should be reliable to set the threshold at 128 x 0.95 s 121. Then the attribution is trivial because we can directly look up the generator instance according to the fingerprint. If the fingerprint is not in the database, it should be a random fingerprint decoded from a real image. We use our immunized model against the combined perturbations in Section 4.5.
Baselines. We compare to two state-of-the woffthhe-rtt art deep fake classifiers (Yu et al., 2019; Wang et al., 2020) as learning-based baselines passively relying inherent visual clues. Because sarring-basee method can only enumerate a finite set of training labels, we consider two scenarios for it: closed world and open world. The difference is whether the testing GAN sources are seen during training or not. This does not matter to our method because ours can work with any N S. 100s For the closed world, we train/evaluate a baseline classifier on 10k/lk images from each of the N + I sources. For the open world, we train N + +1 1-vs-the-othhe other binary classifiers, and predict as "the other" label if and only if all the classifiers predict negative results. We test on 1k images from each of the real sources or N unseen GAN sources. We in addition refer to (Yu et al., 2021) in comparisons as the other proactive but indirect model fingerprinting baseline.
Results. From Table 3 we find:
Closed world #GANs
Open world #GANs
Method
10
100
1
10
100
(1) Deep fake detection and attribu- tion based on our fingerprints per- form equally perfectly (~ 100% ac- curacy) to most of the baselines in the closed world when the number of GAN sources is not too large. How- ever, when N = 100, (Yu et al., 2021) is not applicable due its limited ef- ficiency and scalability. Neither is (Wang et al., 2020) due to its binary classification nature.
(Yu et al., 2019)
0.997
0.998
0.955
0.893
0.102
N/A
(Wang et al., 2020)
0.890
N/A
N/A
0.883
N/A
N/A
(Yu el al., 2021)
1.000
1.000
N/A
1.000
1.000
N/A
Ours
1.000
1.000
1.000
1.000
1.000
1.000
Table 3: Deep fake detection and attribution accuracy on CelebA. A higher value is more desirable. The baseline results are directly copied from (Yu et al., 2021).
(2) Open world also trivial scenario to our method but challenges the baseline classifiers (Yu et al., 2019; Wang et al., 2020). When the number of unseen GAN sources increases to 10, (Yu et al., 2019) even degenerates close to random guess. This is a common generalization issue of the learning- based method. (Yu et al., 2021) is still impractical when N is large.
(3) Since deep fake detection and attribution is a trivial task to our method, it makes our advantages independent of the evolution of GAN techniques. It benefits model tracking and pushes forward the emerging direction of model inventors' responsible disclosure.
5 CONCLUSION
We achieve responsible disclosure of generative models by novel fingerprinting mechanism. It allows scalable ad-hoc generation of a large population of models with distinct fingerprints. We further validate its saturated performance in the deep fake detection and attribution tasks. We appeal to the initiatives of our community to maintain responsible release and regulation of generative models. We hope responsible disclosure would serve one major foundation for AI security.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
The authors strive make this work reproducible. The appendix contains plenty of implementation details. The source code and well-trained models are available at GitHub.
ETHICS STATEMENT
This work does not involve any human subject, nor is our dataset related to any privacy concern.. The authors strictly comply with the ICLR Code of Ethics".
LEKNOWLEDGEMENT
Ning Yu was partially supported by Twitch Research Fellowship. Vladislav Skripniuk was partially supported by IMPRS scholarship from Max Planck Institute. This work was also supported, in part, by the US Defense Advanced Research Projects Agency (DARPA) Media Forensics (MediFor) Program under FA87501620191 and Semantic Forensics (SemaFor) Program under HR001 24.0112. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DARPA. We acknowledge the Maryland Advanced Research Computing Center for providing computing resources. We thank David Jacobs, Matthias Zwicker, Abhinav Shrivastava, and Yaser Yacoob for constructive advice in general.
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring, In USENIX, 2018.
Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: compact facial video forgery detection network. In WIFS, 2018.
Michael Albright and Scott McCloskey. Source generator attribution via inversion. CVPR Workshops, 2019.
Vishal Asnani, Xi Yin, Tal Hassner, and Xiaoming Liu. Reverse engineering of generative models: Inferring model hyperparameters from generated images. arXiv, 2021.
Shumeet Baluja. Hiding images in plain sight: Deep steganography. NeurIPS, 2017.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2018.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. arXiv, 2020.
Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, et The malicious use artificial intelligence: Forecasting, prevention, and mitigation. arXiv, 2018.
Francois Cayre, Caroline Fontaine, and Teddy Furon. Watermarking security: theory and practice. In TSP, 2005.
Huili Chen, Bita Darvish Rouhani, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar, Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In ICMR, 2019.
Robert Chesney and Danielle Citron. Deepfakes and the new disinformation war: The coming age of post-truth geopolitics. Foreign Aff., 2019.
Ingemar Cox, Matthew Miller, Jeffrey Bloom, and Chris Honsinger. Digital watermarking. Springer, 2002.
Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, and Janis Keuper. Unmasking deepfakes with simple features. arXiv, 2019.
10

---

Published as a conference paper at ICLR 2022
Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up-convolution: Can based generative deep neural networks are failing to reproduce spectral distributions. In CVPR, 2020.
Wei Fan, Shruti Agarwal, and Hany Farid. Rebroadcast attacks: Defenses, reattacks, and redefenses. In EUSIPCO, 2018
Han Fang, Weiming Zhang, Hang Zhou, Hao Cui, and Nenghai Yu. Screen-shooting resilient watermarking. TIFS, 2018.
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In ICML, 2020.
Jessica Fridrich. Steganography in digital media: principles, algorithms, and applications. Cambridge University Press, 2009.
Sharath Girish, Saksham Suri, Saketh Rambhatla, and Abhinav Shrivastava. Towards discovery and attribution of open-world gan generated images. In CVPR, 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 2014.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
David Güera and Edward Delp. Deepfake video detection using recurrent neural networks. In AVSS, 2018.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. In NeurlPS, 2017.
Douglas Harris. Deepfakes: False pornography is here and the law cannot protect you. Duke L. & Tech. Rev., 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurlPS, 2017.
Vojtech Holub, Jessica Fridrich, and Tomáš Denemark. Universal distortion function for steganogra- phy in an arbitrary domain. In EURASIP JIS, 2014.
Chih-Chung Hsu, Chia- Chia-Yen Lee, and Yi-Xiu Zhuang. Learning to detect fake face images in the wild. In IS3C, 2018.
Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Lue Gool. Dynamic filter networks. In NeurIPS, 2016.
Xiangui Kang. Jiwu Huang, and Wenjun Zeng. Efficient general print-scanning resilient data hiding based on uniform log-polar mapping. TIFS, 2010.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, 2020.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. 2015.
Daniel and David Megías. Unsupervised steganalysis based on artificial training sets. In EAAI, 2016.
11

---

Published as a conference paper at ICLR 2022
Ke Li and Jitendra Malik. Implicit maximum likelihood estimation. arXiv, 2018.
Zhengzhe Liu, Xiaojuan Qi, Jiaya Jia, and Philip Torr. Global texture enhancement for fake face detection in the wild. In CoRR, 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
Xiyang Luo, Ruohan Zhan, Huiwen Chang, Feng Yang, and Peyman Milanfar. Distortion agnostic deep watermarking. In CVPR, 2020.
Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. Do gans leave artificial fingerprints? In MIPR, 2019.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.
Ding Sheng Ong, Chee Seng Chan, Kam Woh Ng, Lixin Fan, and Qiang Yang. Protecting intellectual property of generative adversarial networks from ambiguity attacks. In CVPR, 2021.
Shelby Pereira and Thierry Pun. Robust template matching for affine resistant image watermarks. TIP, 2000.
Tomáš Pevný, Tomá Filler, and Patrick Bas. Using high-dimensional image models to perform highly undetectable steganography. In /WIHI 2010.
Anu Pramila, Anja Keskinarkaus, and Tapio Seppänen. Increasing the capturing angle in print-cam robust watermarking. Journal of Systems and Software, 2018.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2016.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 2019.
Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: an end-to-end watermarking framework for protecting the ownership of deep neural networks. In ASPLOS, 2019.
Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. Updates-leak: Data set inference and reconstruction attacks online learning. In USENIX, 2020.
Kaushal Solanki, Upamanyu Madhow, BS Manjunath, Shiv Chandrasckaran, and Ibrahim El-Khalil. Print and scan' resilient data hiding in images. TIFS, 2006.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational learning. In NewrIPS, 2017.
Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical pho- tographs. In CVPR, 2020.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin Satoh. Embedding watermarks into deep neural networks. In JCMR, 2017.
Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn images are surprisingly easy to spot... for now. In CVPR, 2020.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv, 2015.
Ning Yu, Larry S Davis, and Mario Fritz. Attributing fake images to gans: Learning and analyzing gan fingerprints. In ICCV, 2019.
Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz. Inclusive gan: Improving data and minority coverage in generative models. In ECCV, 2020.
12

---

Published as a conference paper at ICLR 2022
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In ICCV, 2021.
Wenjia Yuan, Kristin J Dana, Ashwin Ashok, Marco Gruteser, and Narayan Mandayam. Spatially
varying radiometric calibration for camera-display messaging. In GlobalSIP, 2013.
Baiwu Zhang, Jin Peng Zhou, Ilia Shumailov, and Nicolas Papernot. Not my deepfake: Towards plausible deniability for machine-gent machine-generated media. arXiv, 2020.
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Mare Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. CCS Asia, 2018.
Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and simulating artifacts in gan fake images. In WIFS, 2019.
13

---

Published as a conference paper at ICLR 2022
CelebA
LSUN Bedroom
Layer
Bit acc f
FID 4
Bit ace
FID
4x4
0.953
12.06
0.693
21.44
8x8
0.981
12.06
0.950
21.15
16x 16
0.993
11.90
0.935
20.98
32x32
0.991
11.07
0.894
20.24
64
0.972
10.77
0.816
19.85
128 x 128
0,946
10.67
0.805
19.67
Ours (all layers)
0.991
11,50
0.993
20.50
Table 4: Fingerprint detection in bitwise accuracy and generation fidelity in FID w.r.t the layer to modulate fingerprints. 1/ indicates a higher/lower value is more desirable.
A APPENDIX
A.1 IMPLEMENTATION DETAILS
We set the length of latent code d: 512. We non-trivially select the number of bits for fingerprint de 128 according to the analysis study on the capacity Section 4.2. Our encoder E is composed of 8 fully-connected neural layers followed by LeakyReLU nonlinearity, the same as the mapping network in StyleGANZ (Karras et al., 2020). The generator G is almost the same as that in StyleGAN2 except we input the latent code through the input end (replacing the learnable constant tensor) rather than through the modulation. The discriminator D is the same as that in StyleGAN2. The decoder almost the same as the discriminator except the output size adapted to the latent code size plus the fingerprint size d₂ + de-
W
We train our model using Adam optimizer (Kingma Ba, 2015) for 400 epochs. We use no exponential decay rate (31 0.0) for the first-moment estimates, and the exponential decay rate 32 = 0.99 for the second-moment StyleGAN2. We train NVIDIA V100 GPUs with 16GB of memory each. Based on the memory available and the training performance, we set the batch size at 32. The training lasts for about 6 days.
moment estimates. The learning rate 7j = 0.002, the same as that in
Our code is modified from the GitHub repository of StyleGAN2 (Karras al., 2020) official TensorFlow implementation (config E)?. All the environment requirements, dependencies, data preparation, and command-line calling conventions exactly inherited.
A.2 ABLATION STUDY ON MODULATION
In an ablation study, we investigate the effectiveness of fingerprint detection and fidelity of generation when modulating fingerprint embeddings to different generator layers (resolutions).
From Table 4 find:
(1) For effectiveness, the optimal single layer to modulate fingerprints appears one the middle layers, specific to datasets: 16x 16 for CelebA and 8x8 for LSUN Bedroom. But our all-layer modulation can achieve comparable or better performance. This should be consistent with different datasets because fingerprint detection turns more effective when we encode fingerprints to more parts of the generator.
(2) For fidelity, the side effect of fingerprinting is less significant if modulation happens in the shallower layer. This is because fingerprinting and generation are distinct tasks, and a shallower modulation leads to less crosstalk. However, considering the FID variance is not significant in general, we regard all-layer modulation as a desirable trade-off between effectiveness and fidelity.
A.3 ABLATION STUDY ON LOSS TERMS
In another ablation study, on CclebA we investigate the contribution of each loss term in Equation towards fingerprint detection and generation fidelity.
From Table 5 we find:
hhtpps:/ comb.ccm//wv/
14

---

Published as a conference paper at ICLR 2022
Loss configuration
Bit acc
FID
Lad Lati Ayhe An Lati + Aahe + A4hoou
9.37
0.999
12.84
0,988
12.24
Aahat + Nahe + Xahcou X2h
0.991
11.50
Table 5: Fingerprint detection in bitwise accuracy and generation fidelity in FID w.r.t loss configura- tion on CelebA. ft/ indicates higher/lower value is more desirable.
(I) For effectiveness, comparing between the first and second rows, solely the fingerprint reconstruc- tion loss term L. itself is effective enough enable saturated fingerprint detection performance.
(2) Comparing between the second and third rows, the consistency loss term Leonst improves FID due to the explicit disentanglement learning between latent code and fingerprint representations.
(3) Comparing between the third and fourth rows, the latent code reconstruction term Lg implicitly improves FID due to its benefit to avoid mode collapse (Srivastava et al., 2017). The collaboration between fingerprint reconstruction and latent code reconstruction through the same decoder further facilitates the disentanglement learning.
A.4 MORE GENERATED SAMPLES WITH FINGERPRINTS
We show in Figure more samples LSUN Bedroom and LSUN Cat with high fidelity and disentangled controls of latent code and fingerprint code.
A.5 FINGERPRINT VISUALIZATION
In order to visualize the imperceptibility and effectiveness of our fingerprinting, in Figure 7 Top generate images given the same latent code and gradually-varying fingerprint codes. The differences among these images imperceptible, which supports the fidelity of our generation, and the disen- tangled control between latent code and fingerprint code. In Figure Bottom we demonstrate the residual images
the lest-most image. Because the original differences are imperceptible, have to magnify the difference values x to visualize the fingerprint patterns. Our detector effectively learns attribute different generator sources based such faint but distinguishable patterns.
Fingerpiit Fingerpiin
Fingarprinn
Fingarprint
Firqarprint
Fingerprint
Finganprint
Fingarprint Fingerprint
Fingurpeint
Latent code Lafent code
Latend code
Latent code
Laterd code
Latent code Ladent Latentcode Latent code Latend code
(a) LSUN Bedroom 128x
(b) LSUN Cat 256x
Figure 6: Fidelity and disentangled control: generated samples from five generator instances. For each row, unique fingerprint instantiate generator. For each column, feed in the same latent code to the generator instances.
15

---

Published as a conference paper at ICLR 2022
5x residual w.r.f. Fingerprinted left-most image generation
Fingerprint
Fingerprint 2
Figure 7: Visualization for the imperceptible patterns of fingerprints. Top: generated images given the same latent code and yrauualy-varrying fingerprint codes from "fingerprint I" to "fingerprint 2". Bottom: residual images w.r.t the lest-most image. Because the differences are too faint, we magnify the pixel values x5 for clear visualization of the fingerprint patterns.
CelchA p-value U
LSUN Bedroom
Method
Bit acc f
FID |
Bit acc fh p-value Y
FID 1
SNGAN
15.88
26.76
Ours
0.991
10
15.88
0.986
< 10-38 34
28.76
WGAN-GP
12.16
21.13
Ours
0.989
10
13.07
0.964
10-3a
22.89
Table 6: Fingerprint detection in bitwise accuracy with p-value to accept the null hypothesis test, and generation fidelity in FID. 1/4/ indicates higher/lower value is more desirable.
A.6 DIFFERENT GENERIC GAN MODELS
Our fingerprinting design is agnostic generic GAN models. We replace the StyleGAN2 backbone with SNGAN (Miyato et al., 2018) or WGAN-GP (Gulrajani et al., 2017) backbone, and report our fingerprint bitwise detection accuracy and fidelity CelebA and LSUN Bedroom datasets.
From Table 6 we find:
(1) Our fingerprinting is generalizable varying GAN models with near-perfed detection accuracy with p-value close to zero, regardless of the original performance of the models.
(2) Our fingerprinting results in negligible 2.01 FID degradation. This is worthy trade-off to introduce the fingerprinting function. In particular for SNGAN on CelebA, our solution does not hurt the original generation quality at all.
16


Reviews:
[{"review_id": "u3QnwtxhTp1", "reviewer": "Reviewer_RiQf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea to actively protect generated synthetic data for bad use is certainly interesting, however I believe this paper cannot be published in its present form since in my opinion there are several issues as described below.\n\nContribution. The concept of fingerprinting generative models is not new. This has been already recently proposed at ICCV (Yu et al. 2021). For my understanding the main concept is the same, i.e. embedd a fingerprint in the model and enable deepfake detection and attribution by decoding the fingerprints from the generated images. Also note that in Yu et al. authors talk about a proactive and responsible disclosure of pre-trained GAN models. In this respect I find that the first contribution claimed in this submission is not novel. The main difference for my understanding is that the method proposed in Yu et al. cannot scale up to a large number of fingerprints differently from this work. Scalability is hence the main contribution with respect to state-of-the-art, given that performance are very much similar to those obtained by Yu et al. (see experimental results). I think this is not sufficient for a publication at ICLR. It is also worth noting that using an autoencoder-based method to encode a fingerprint into a classifier has been already done in the literature.\n\nRelated work. The section about related work is confused in my opinion. In the forensics field watermarking and steganography are two research topics with different goals. Even if they both aim at hiding information, the way this is conducted is fundamentally different. In this regard, I would not mix them in a single paragraph. I believe what is done in this work is more similar to watermarking and the innovation with respect to current literature should be better highlighted. For what concerns network watermarking it is not true that a solution for generative models is missing, see the recent work (Ong et al., CVPR 2021). \n\nExperimental analysis\n\n- Experiments are carried out only on StyleGAN2 model. What happens for different GAN architectures? No generalization analysis is conducted. \n\n- The proposed approach can identify more than 10^36 generator instances, but are these perfectly distinguishable from istances of other generators? This is not clear and should be demonstrated.\n\n- Robustness study is also not sufficient. It is not clear for example if adding adversarial noise can destroy important traces of the fingerprint. More specifically, the ability of the attacker to break the fingerprint has not been studied thoroughly (see Section 4.4.)\n\n- I am very much confused by the experiments carried out in Section 4.6. First, I cannot understand why comparing with passive methods (Wang et al., 2020). Then, for attribution based methods, only one approach has been tested (Yu et al. 2019) while neglecting many others (see references below). Finally, why not comparing with watermarking based methods? It would be very important to show that the proposed method can achieve much better performance than other active approaches. \nIn this same section authors talk about N GAN sources, I imagine this means the same architecture but a different trained model. This should be made clear, because it can create confusion.\n\n- There are four loss terms in eq.5. I cannot find an ablation study in the experimental section that shows the importance of such terms.\n\n- This work aims at solving the problem of deepfake misuse. Now, I understand that it is possible to perform attribution for a known GAN model. However it is not clear what happens if the synthetic image has not been fingerprinted. Suppose for example that the bad actor builds its own generator without a fingerprint. Will the method confuse the image with a real one? No experiments have been presented to understand the behaviour in this scenario.\n\n=========== Post rebuttal comments ===========\n\nI want to thank the authors for answering to all my comments. Many important points have been clarified\nand the paper has been updated accordingly. In particular, I appreciate the new experiments on other GAN architectures\nand the ablation study on different loss term configurations. I also find that now the connections with state-of-the-art are much more clear.\nIn addition, the main contribution as well as the scenario of interest have been better explained. For these reasons I increase my score to borderline accept.\n\nReferences\n\n- Ong et al. Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attacks, CVPR 2021\n- Albright and McCloskey, Source Generator Attribution via Inversion, CVPR Workshop 2019\n- Asnani et al. Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images, arXiv 2021\n- Girish et al. Towards Discovery and Attribution of Open-world GAN Generated Images, CVPR 2021\n", "summary_of_the_paper": "Aim of this work is to develop a method to fingerprint GAN models. In this way, images generated from that model can be detected and attributed to a specific GAN model. This would help in a scenario where a malicious actor uses a published GAN model to produce fake images. In fact, the inserted fingerprint would allow to identify the model and hence to establish that the image is synthetic. From a technical point of view this is achieved by adding a 128-bit fingerprint and jointly training an auto-encoder in a GAN framework. Experiments shows that the proposed approach turns out to be an efficient and scalable solution if compared with state-of-the-art.\n", "summary_of_the_review": "My main concerns about this work are related to the contribution, which in my opinion is too limited with respect to state-of-the-art, and also to the insufficient experimental validation. I think that more work is needed in terms of experiments to support the claims, in particular about the generalization, uniqueness and robustness of the fingerprints. For these reasons I believe that this submission is not ready to be published.\n\nUPDATES\nAuthors have better motivated their proposed method with respect to current literature and also added more experiments, that makes the proposal more convincing. I still believe that some more work on the uniqueness and robustness of the fingerprints should be done, however this can be carried out in a future work. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YUcZcN0X2WV", "reviewer": "Reviewer_sqeb", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1.\tThis work is significant because it can provide a potential way to detect the deep fake images and attribute the responsibility to a particular user who generated the fake images. Theoretically, this model only needs to be trained once, and a large population of models can be instantiated, each of which has a generator modulated using a different fingerprint.  By doing so, this model has the potential to improve scalability than existing methods that train an auto-encoder to fingerprint the images in the training set and train a GAN for each fingerprint.\n2.\tThis work has technical novelty in that it defines a new loss function for StyleGAN2. This loss function contains a loss term that corresponds to the correctness of the decoded fingerprint, and a loss term corresponds to the consistency of the images generated using the same latent code but different fingerprints. The technical novelty of this work also lies in that this work changed the architecture of the StyleGAN2 backbone by modulating the convolutional filters in the generator with the fingerprint embedding. \n3.\tThis work evaluated the performance of the proposed model using three different datasets. The results are compared with a baseline from the existing work (Yu et al., 2021) that relies on fingerprinting the images in the training data. The experiments are carefully designed, and the results showed that the proposed method could provide the same level of fingerprint detection accuracy and maintain the same level of quality in the generated images as the model from Yu et al., 2021. \n\nWeaknesses\n1.\tThe proposed model is tied to the StyleGAN2 model, while the baseline method (Yu et al. 2021) is agnostic to models. It would be helpful if the authors could demonstrate how the same mechanism can work with other GAN models;\n2.\tHow the scalability of the model is demonstrated can be improved. As shown in section 4.3, the authors only provided the relationship between the detection accuracy and the training set size. The results suggest that there need to be at least 10k samples in the training set to reach high fingerprint detection accuracy in the testing set. It remains unclear if the baseline method, which fingerprints the images of the training set, also requires the same number of or more samples. It is possible that the baseline method requires a smaller sample size than the proposed method. It is also unclear how the training time of the proposed model is compared to the baseline model. It would be helpful if the authors could provide some information concerning the sample size required for the baseline method and the training time required for the baseline method and the proposed method. Alternatively, the authors can provide a brief explanation on this matter. \n3.\tEven though the proposed method can allow for initiating a large number of models with different fingerprints efficiently once a model is trained, training the model requires at least 10k samples from the fingerprint space. This indicates that even if the model creator only needs to release a small number of models, the training will still need to be conducted to a large amount of fingerprints. If the authors can provide some additional insights about this issue that would be very helpful.\nOther comments:\n1.\tIt would be helpful to add some discussion on comparing the robustness and immunizability of the proposed method and the method proposed in Yu et al. 2021. \n2.\tIt seems like the baseline results are directly obtained from the published manuscript; it would be helpful to make this clear.\n", "summary_of_the_paper": "This work aims to provide a method to tackle deep fake by making the samples generated using the generative adversarial networks (GANs) contain a user-specific fingerprint that can be accurately detected and attributed to each user.  \n\nThis paper built and evaluated a GAN model upon the StyleGAN2 backbone. The generator in the proposed GAN is modulated with different fingerprints on the fly. In other words, after training one model with randomly sampled fingerprints and latent code, a large set of models modulated with different fingerprints can be generated within seconds. The experiments results show that the fingerprint can be accurately detected from the GAN generated images, and the fingerprint has a negligible side effect on the original generation quality. The experiments also demonstrated that the optimal number of unique fingerprints this model can accommodate is over 2^128.\n", "summary_of_the_review": "Based on the significance and novelty of this work, the reviewer’s recommendation is “Accept”.            ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OMnjucqrsJy", "reviewer": "Reviewer_yqaN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Summary\nIn this paper, the authors address the problem of fingerprinting GAN (StyleGAN2 in the proposed setting) in a scalable way, e.g. after initial processing, arbitrary fingerprints can be added. In doing so, the authors proposed to leverage modulated convolution layers which scales the output of GAN’s intermedia layers by projection embedded in a channel wise fashion. The authors also provide a set of metrics to evaluate GAN fingerprinting techniques, and accordingly demonstrate the superiority of the proposed method.\n\n# Strong Points\n\n- This paper is well motivated to address the important problem of fingerprinting generative models that are becoming increasingly powerful recently such that provdenting, or at least identifying, its ill-use.\n- Overall, this paper is well-structured and easy to follow.\n- This paper presents a scalable way of fingerprinting GAN, where after one pass of training, a lot of fingerprints can be added requiring negligible additional resources.\n- This paper presents a set of metrics for evaluating the tasks of GAN fingerprinting, which is important as such tasks are still new. Using these metrics, experiments show the advantage of the proposed method.\n\n# Weak Points\n- There is no visualization showing the effect of channel-wise modulation as a fingerprinting technique.. It would be helpful for readers to understand such channel-wise effects using visualization, for example like that in [1].\n- The experiments are only done with resolutions 128x128 and 256x256. As modern GANs can generate much higher resolution (say, 1024x1024) with fine details, knowing the proposed method’s performance in higher resolution can help justify its motivation dealing with modern GANs.\n- In Eq (5) there are four coefficients “ to balance the magnitude of each loss term.“ but how  are they chosen and what is the impact of such a choice are not elaborate? It would be helpful to provide analysis to the choice, as the readers may want to apply the proposed technique to other GAN models which likely have different loss terms and thus magnitudes. \n\n# Questions to Authors\nAlthough this work as presented is interesting, there are some questions I have from reading it thoroughly. They are:\n- In Page 3, the author claims that “they can barely sustain a long time against the adversarial iterations of GAN techniques.” Is there any concrete example of fingerprinting tech **not** withstanding adversarial iteration of GANs?\n- It seems that the loss term $L_z$ in Eq (2) is orthogonal to other loss terms and is less connected to them. What’s the motivation behind using it besides it’s simply a nice to have from other recent works? It is a must-to-have ingredient in the whole pipeline?\n\n\n# Other Comments\n- As $L_z$ and $L_c$ in Eq (2) and Eq (3) respectively are calculated with the same decoder $F$, it seems to imply that they are calculated on different parts of the logits. It’s better to explicitly mention this fact to avoid confusion.\n\n# Assessment\nOverall, I think the strong points( the motivation, novel technique, and important set of metrics for such tasks that follow-up works in this direction would find useful.) are important and overweight weak points (there are more like better to know). So I would recommend a clear acceptance, although the authors are enraged to address may questions/concerns to make the draft more comprehensive.\n\n\n# Ref\n[1] Karras et al., 2021: alias-free generative adversarial networks\n\n\n# Post-rebuttal\n\nI would like to thank the authors for their response, in which most of my issues are solved.\n", "summary_of_the_paper": "In this paper, the authors address the problem of fingerprinting GAN (StyleGAN2 in the proposed setting) in a scalable way, e.g. after initial processing, arbitrary fingerprints can be added. In doing so, the authors proposed to leverage modulated convolution layers which scales the output of GAN’s intermedia layers by projection embedded in a channel wise fashion. The authors also provide a set of metrics to evaluate GAN fingerprinting techniques, and accordingly demonstrate the superiority of the proposed method.", "summary_of_the_review": "Overall, I think the strong points( the motivation, novel technique, and important set of metrics for such tasks that follow-up works in this direction would find useful.) are important and overweight weak points (there are more like better to know). So I would recommend a clear acceptance, although the authors are enraged to address may questions/concerns to make the draft more comprehensive.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Dv3GKwmG_s0", "reviewer": "Reviewer_GMRc", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper addresses an interesting problem, grounded on concrete and impactful issues raised by the emergence of deep fakes. The authors improve on the literature by addressing the main limitation of previous works, namely the scalability of fingerprinting a lot of generated data. The solution presented in the paper is clear, and the soundness of the approach is well grounded on empirical verifications. I particularly liked that everything I was wondering through my first reading of the paper was eventually addressed somewhere later in the text, either in discussions, or through empirical evidence. Overall, and to the best of my knowledge, I have not found any shortcoming in the paper, although I am not an expert in fingerprinting for GANs.\nRegarding the related works, a search on Scholar, Scopus, and DBLP seems to indicate that no major reference is missing regarding the core topic of this submission.\n\nHereafter are some thoughts & suggestions to challenge this submission a bit.\n\nMaybe, I wish the final section (§5 conclusion) would have brought more discussions & perspectives about potential limitations of the current approach, and what could remain to be done. As an example, I guess from the paper that the authors' approach mostly relies on the assumption that only a few (legitimate) entities are able to build and train state-of-the-art GANs. I may be somewhat speculative due to my lack of expertise in GANs, but I guess that as long as the cost for building such models will decrease, more malicious entities will be able to build their models, thereby mitigating the impact of fingerprinting. It might be valuable to add a short comment in this sense, even if this does not question the good quality of this submission. \n\nLikewise, the model inventor is assumed to be honest here. This naturally goes beyond the scope of this paper, but maybe discussing further works about non-repudiation could represent a nice open question towards the responsible release of industry-wide models.\n\nA few minor remarks can be found hereafter. \n\n**Sec 3.1**\n* The parameters $\\lambda_i$ are arbitrarily fixed to similar values in order of magnitude, which is not discussed. While I guess that this is beyond the scope of this submission, is there any reason why all the $\\lambda_i$ coefficient of the final objective loss should have the same order of magnitude? \n\n**Sec 4.1** \n* \"Our method results in negligibly < 2.93 FID degrading. This is a reasonable trade-off between effectiveness and fidelity.\" -> could you elaborate a bit more ? More precisely, are there any reference values for FID with which the non-advised reader may compare ? \n\n**Sec 4.2** \n* \"on one hand\" -> \"on the one hand\" \n* $2^{128 \\times 0.991} \\approx 10^{38}$ instead of 36 (even better). \n\n**Sec 4.4** \n* The description of experiments for fingerprint presence/value attacks are clear, but remain very high-level. I have not found any detail about the implementation and training of the CNN binary classifier, nor any raw performance data. Accordingly, it does not enable us to assess whether enough efforts have been done to reach better accuracy, which may invalidate the secrecy claims. Could you elaborate a bit more on this ? \n\n## Addendum: major overlaps between the Related Works sections of this submission and (Yu et al., 2021)\nSince this submission compares to (Yu et al., 2021) a lot, I quickly read this reference and I have found some major overlaps in the related work section (up to minor updates). To the best of my knowledge, this only concerns the related works section. Nevertheless, this may still technically represent some plagiarism. Hence, **I urge the authors to rewrite this section, or at the very least to mention that the related works section is mostly taken from (Yu et al., 2021)**. \n", "summary_of_the_paper": "This paper proposes a technique to allow inventors of generated data (e.g. deepfakes) to mark any generated data that may be broadcast through a public API (e.g. like OpenAI did), so that the inventor can easily assess whether a data has been generated through its generative model, and therefore mitigate harmful effects of such fake data. The core idea resides in ploughing a unique fingerprint into the generative model, making it somehow unique for each generated data. The generative model is jointly trained with a decoder whose aim is, given a potentially fake data, to recognize which fingerprint has been used to generate the data. The authors argue that their technique does not cause too much computational overhead, occurs negligible performance degradation compared to standard generative models, and is scalable and robust against potential attacks aiming at circumventing fingerprinting. The claims are verified on several public, GANs-oriented, datasets.", "summary_of_the_review": "This paper addresses an interesting problem about fingerprinting data generated by GANs, in order to be able to trace any misuse of deepfake. The paper improves state of the art by addressing a major shortcoming of previous works. Overall, the presented solution represents a clear impact on industry-deployed GANs. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6PhSFvML5KD", "reviewer": "Reviewer_fu3Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\nProactively fingerprinting a model is a clever idea. The advantage of this method is that the model inventor just needs to perform 1 training phase and can deploy plenty models hard-coding random fingerprint to end users. Extensive experiments are reported justifying the choice of the fingerprint code length and the design of the modulated convolution layer.\n\nThe paper is well written and easy to follow.\n\nCons: \n(i) My main concern is that the definition of detection and attribution of synthesised images is somewhat misleading. \"Detection\" usually means if an image is real or generated from *any* GAN models. \"Attribution\" refers to predicting which GAN source generates a synthesis image. However, in this paper the authors experiment with StyleGAN2 only and the \"GAN sources\" are actually from a single StyleGAN2 model perturbed with different fingerprint codes. This makes the comparison with Yu 2019 and Wang 2020 a bit unfair. It also raises several questions of whether the proposed method work on other GAN architectures such as SNGAN, ProGAN, etc, also how the fingerprint code could be tracked if each model is trained this way.\n\n(ii) The application side of the proposed method is rather limited, as it requires the model creator to cooperate, or being responsible, from the beginning. It does not solve the current deepfake problem (strictly speaking, GAN synthesised images are not deepfake which is a different terminology). Nevertheless, the authors carefully state a user case that motivates their approach in the Introduction section. While I do not disagree with the authors, I feel the narrow application is a limitation of their proactive approach.\n\n(iii) The authors use bit-wise accuracy to evaluate the fingerprint detection performance throughout their paper. I think there should be an additional metric that is stricter than bit-wise accuracy: a binary metric that evaluates to True if the exact fingerprint code is predicted. This is to evaluate the reliability of the detection model, since 1 bit difference in the detection can lead to a different fingerprint code (similar to cryptographic hash which does not have metric property).\n\n(iv) (Minor). Fig5-b, can the authors elaborate why the immunized model perform worse than the original model when exposed to blurring perturbation? ", "summary_of_the_paper": "This paper addresses the detection and attribution of synthesised images to its GAN sources. The authors propose a proactive method to inject fingerprint into GAN model's parameters during its training. This is done via a modulated convolution layer to integrate the fingerprint into its filter weight and an additional loss term to predict the fingerprint code from the generator's output image. The authors demonstrated that the model fingerprint can be detected from its generated images with high confidence while retaining the synthesis fidelity. The proposed approach is shown robust against common image perturbation sources if these perturbations are shown during training. It also achieves perfect detection and attribution scores when compared with existing methods.", "summary_of_the_review": "Overall the paper introduce an interesting approach to address the problem of synthesised image detection/attribution. Although the proposed method certainly has merits and applications, I feel that the paper has slightly different terminology of image detection/attribution versus the literature. At the current state of the paper, I am at borderline but would like to hear the authors to elaborate on my points raised above.\n\nUpdate: I have read the authors' response and other reviews. I think the main advantage of this paper is a novel strategy to fingerprint a GAN model, while the main drawback is that it works on multiple instances of a single GAN source, instead of multiple GAN sources like the baseline approaches. I strongly encourage the authors to clarify this in their next revision.\n\nI upgrade my decision to borderline accept.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes and studies a method for the responsible disclosure of a fingerprint along with samples generated by a generative model, which has important applications in identifying "deep fakes". The authors establish both the detectability of their fingerprint-without significant loss of fidelity-as well as the robustness to perturbations. The reviewers found the problem and contributions to be important and significant, well substantiated by an extensive experimental study.
Decision: Accept (Spotlight)