Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the detection and attribution of synthesised images to its GAN sources. The authors propose a proactive method to inject fingerprint into GAN model's parameters during its training. This is done via a modulated convolution layer to integrate the fingerprint into its filter weight and an additional loss term to predict the fingerprint code from the generator's output image. The authors demonstrated that the model fingerprint can be detected from its generated images with high confidence while retaining the synthesis fidelity. The proposed approach is shown robust against common image perturbation sources if these perturbations are shown during training. It also achieves perfect detection and attribution scores when compared with existing methods.
main_review: Pros:
Proactively fingerprinting a model is a clever idea. The advantage of this method is that the model inventor just needs to perform 1 training phase and can deploy plenty models hard-coding random fingerprint to end users. Extensive experiments are reported justifying the choice of the fingerprint code length and the design of the modulated convolution layer.

The paper is well written and easy to follow.

Cons: 
(i) My main concern is that the definition of detection and attribution of synthesised images is somewhat misleading. "Detection" usually means if an image is real or generated from *any* GAN models. "Attribution" refers to predicting which GAN source generates a synthesis image. However, in this paper the authors experiment with StyleGAN2 only and the "GAN sources" are actually from a single StyleGAN2 model perturbed with different fingerprint codes. This makes the comparison with Yu 2019 and Wang 2020 a bit unfair. It also raises several questions of whether the proposed method work on other GAN architectures such as SNGAN, ProGAN, etc, also how the fingerprint code could be tracked if each model is trained this way.

(ii) The application side of the proposed method is rather limited, as it requires the model creator to cooperate, or being responsible, from the beginning. It does not solve the current deepfake problem (strictly speaking, GAN synthesised images are not deepfake which is a different terminology). Nevertheless, the authors carefully state a user case that motivates their approach in the Introduction section. While I do not disagree with the authors, I feel the narrow application is a limitation of their proactive approach.

(iii) The authors use bit-wise accuracy to evaluate the fingerprint detection performance throughout their paper. I think there should be an additional metric that is stricter than bit-wise accuracy: a binary metric that evaluates to True if the exact fingerprint code is predicted. This is to evaluate the reliability of the detection model, since 1 bit difference in the detection can lead to a different fingerprint code (similar to cryptographic hash which does not have metric property).

(iv) (Minor). Fig5-b, can the authors elaborate why the immunized model perform worse than the original model when exposed to blurring perturbation? 
summary_of_the_review: Overall the paper introduce an interesting approach to address the problem of synthesised image detection/attribution. Although the proposed method certainly has merits and applications, I feel that the paper has slightly different terminology of image detection/attribution versus the literature. At the current state of the paper, I am at borderline but would like to hear the authors to elaborate on my points raised above.

Update: I have read the authors' response and other reviews. I think the main advantage of this paper is a novel strategy to fingerprint a GAN model, while the main drawback is that it works on multiple instances of a single GAN source, instead of multiple GAN sources like the baseline approaches. I strongly encourage the authors to clarify this in their next revision.

I upgrade my decision to borderline accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work aims to provide a method to tackle deep fake by making the samples generated using the generative adversarial networks (GANs) contain a user-specific fingerprint that can be accurately detected and attributed to each user.  

This paper built and evaluated a GAN model upon the StyleGAN2 backbone. The generator in the proposed GAN is modulated with different fingerprints on the fly. In other words, after training one model with randomly sampled fingerprints and latent code, a large set of models modulated with different fingerprints can be generated within seconds. The experiments results show that the fingerprint can be accurately detected from the GAN generated images, and the fingerprint has a negligible side effect on the original generation quality. The experiments also demonstrated that the optimal number of unique fingerprints this model can accommodate is over 2^128.

main_review: Strengths
1.	This work is significant because it can provide a potential way to detect the deep fake images and attribute the responsibility to a particular user who generated the fake images. Theoretically, this model only needs to be trained once, and a large population of models can be instantiated, each of which has a generator modulated using a different fingerprint.  By doing so, this model has the potential to improve scalability than existing methods that train an auto-encoder to fingerprint the images in the training set and train a GAN for each fingerprint.
2.	This work has technical novelty in that it defines a new loss function for StyleGAN2. This loss function contains a loss term that corresponds to the correctness of the decoded fingerprint, and a loss term corresponds to the consistency of the images generated using the same latent code but different fingerprints. The technical novelty of this work also lies in that this work changed the architecture of the StyleGAN2 backbone by modulating the convolutional filters in the generator with the fingerprint embedding. 
3.	This work evaluated the performance of the proposed model using three different datasets. The results are compared with a baseline from the existing work (Yu et al., 2021) that relies on fingerprinting the images in the training data. The experiments are carefully designed, and the results showed that the proposed method could provide the same level of fingerprint detection accuracy and maintain the same level of quality in the generated images as the model from Yu et al., 2021. 

Weaknesses
1.	The proposed model is tied to the StyleGAN2 model, while the baseline method (Yu et al. 2021) is agnostic to models. It would be helpful if the authors could demonstrate how the same mechanism can work with other GAN models;
2.	How the scalability of the model is demonstrated can be improved. As shown in section 4.3, the authors only provided the relationship between the detection accuracy and the training set size. The results suggest that there need to be at least 10k samples in the training set to reach high fingerprint detection accuracy in the testing set. It remains unclear if the baseline method, which fingerprints the images of the training set, also requires the same number of or more samples. It is possible that the baseline method requires a smaller sample size than the proposed method. It is also unclear how the training time of the proposed model is compared to the baseline model. It would be helpful if the authors could provide some information concerning the sample size required for the baseline method and the training time required for the baseline method and the proposed method. Alternatively, the authors can provide a brief explanation on this matter. 
3.	Even though the proposed method can allow for initiating a large number of models with different fingerprints efficiently once a model is trained, training the model requires at least 10k samples from the fingerprint space. This indicates that even if the model creator only needs to release a small number of models, the training will still need to be conducted to a large amount of fingerprints. If the authors can provide some additional insights about this issue that would be very helpful.
Other comments:
1.	It would be helpful to add some discussion on comparing the robustness and immunizability of the proposed method and the method proposed in Yu et al. 2021. 
2.	It seems like the baseline results are directly obtained from the published manuscript; it would be helpful to make this clear.

summary_of_the_review: Based on the significance and novelty of this work, the reviewer’s recommendation is “Accept”.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Aim of this work is to develop a method to fingerprint GAN models. In this way, images generated from that model can be detected and attributed to a specific GAN model. This would help in a scenario where a malicious actor uses a published GAN model to produce fake images. In fact, the inserted fingerprint would allow to identify the model and hence to establish that the image is synthetic. From a technical point of view this is achieved by adding a 128-bit fingerprint and jointly training an auto-encoder in a GAN framework. Experiments shows that the proposed approach turns out to be an efficient and scalable solution if compared with state-of-the-art.

main_review: The idea to actively protect generated synthetic data for bad use is certainly interesting, however I believe this paper cannot be published in its present form since in my opinion there are several issues as described below.

Contribution. The concept of fingerprinting generative models is not new. This has been already recently proposed at ICCV (Yu et al. 2021). For my understanding the main concept is the same, i.e. embedd a fingerprint in the model and enable deepfake detection and attribution by decoding the fingerprints from the generated images. Also note that in Yu et al. authors talk about a proactive and responsible disclosure of pre-trained GAN models. In this respect I find that the first contribution claimed in this submission is not novel. The main difference for my understanding is that the method proposed in Yu et al. cannot scale up to a large number of fingerprints differently from this work. Scalability is hence the main contribution with respect to state-of-the-art, given that performance are very much similar to those obtained by Yu et al. (see experimental results). I think this is not sufficient for a publication at ICLR. It is also worth noting that using an autoencoder-based method to encode a fingerprint into a classifier has been already done in the literature.

Related work. The section about related work is confused in my opinion. In the forensics field watermarking and steganography are two research topics with different goals. Even if they both aim at hiding information, the way this is conducted is fundamentally different. In this regard, I would not mix them in a single paragraph. I believe what is done in this work is more similar to watermarking and the innovation with respect to current literature should be better highlighted. For what concerns network watermarking it is not true that a solution for generative models is missing, see the recent work (Ong et al., CVPR 2021). 

Experimental analysis

- Experiments are carried out only on StyleGAN2 model. What happens for different GAN architectures? No generalization analysis is conducted. 

- The proposed approach can identify more than 10^36 generator instances, but are these perfectly distinguishable from istances of other generators? This is not clear and should be demonstrated.

- Robustness study is also not sufficient. It is not clear for example if adding adversarial noise can destroy important traces of the fingerprint. More specifically, the ability of the attacker to break the fingerprint has not been studied thoroughly (see Section 4.4.)

- I am very much confused by the experiments carried out in Section 4.6. First, I cannot understand why comparing with passive methods (Wang et al., 2020). Then, for attribution based methods, only one approach has been tested (Yu et al. 2019) while neglecting many others (see references below). Finally, why not comparing with watermarking based methods? It would be very important to show that the proposed method can achieve much better performance than other active approaches. 
In this same section authors talk about N GAN sources, I imagine this means the same architecture but a different trained model. This should be made clear, because it can create confusion.

- There are four loss terms in eq.5. I cannot find an ablation study in the experimental section that shows the importance of such terms.

- This work aims at solving the problem of deepfake misuse. Now, I understand that it is possible to perform attribution for a known GAN model. However it is not clear what happens if the synthetic image has not been fingerprinted. Suppose for example that the bad actor builds its own generator without a fingerprint. Will the method confuse the image with a real one? No experiments have been presented to understand the behaviour in this scenario.

=========== Post rebuttal comments ===========

I want to thank the authors for answering to all my comments. Many important points have been clarified
and the paper has been updated accordingly. In particular, I appreciate the new experiments on other GAN architectures
and the ablation study on different loss term configurations. I also find that now the connections with state-of-the-art are much more clear.
In addition, the main contribution as well as the scenario of interest have been better explained. For these reasons I increase my score to borderline accept.

References

- Ong et al. Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attacks, CVPR 2021
- Albright and McCloskey, Source Generator Attribution via Inversion, CVPR Workshop 2019
- Asnani et al. Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images, arXiv 2021
- Girish et al. Towards Discovery and Attribution of Open-world GAN Generated Images, CVPR 2021

summary_of_the_review: My main concerns about this work are related to the contribution, which in my opinion is too limited with respect to state-of-the-art, and also to the insufficient experimental validation. I think that more work is needed in terms of experiments to support the claims, in particular about the generalization, uniqueness and robustness of the fingerprints. For these reasons I believe that this submission is not ready to be published.

UPDATES
Authors have better motivated their proposed method with respect to current literature and also added more experiments, that makes the proposal more convincing. I still believe that some more work on the uniqueness and robustness of the fingerprints should be done, however this can be carried out in a future work.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a technique to allow inventors of generated data (e.g. deepfakes) to mark any generated data that may be broadcast through a public API (e.g. like OpenAI did), so that the inventor can easily assess whether a data has been generated through its generative model, and therefore mitigate harmful effects of such fake data. The core idea resides in ploughing a unique fingerprint into the generative model, making it somehow unique for each generated data. The generative model is jointly trained with a decoder whose aim is, given a potentially fake data, to recognize which fingerprint has been used to generate the data. The authors argue that their technique does not cause too much computational overhead, occurs negligible performance degradation compared to standard generative models, and is scalable and robust against potential attacks aiming at circumventing fingerprinting. The claims are verified on several public, GANs-oriented, datasets.
main_review: This paper addresses an interesting problem, grounded on concrete and impactful issues raised by the emergence of deep fakes. The authors improve on the literature by addressing the main limitation of previous works, namely the scalability of fingerprinting a lot of generated data. The solution presented in the paper is clear, and the soundness of the approach is well grounded on empirical verifications. I particularly liked that everything I was wondering through my first reading of the paper was eventually addressed somewhere later in the text, either in discussions, or through empirical evidence. Overall, and to the best of my knowledge, I have not found any shortcoming in the paper, although I am not an expert in fingerprinting for GANs.
Regarding the related works, a search on Scholar, Scopus, and DBLP seems to indicate that no major reference is missing regarding the core topic of this submission.

Hereafter are some thoughts & suggestions to challenge this submission a bit.

Maybe, I wish the final section (§5 conclusion) would have brought more discussions & perspectives about potential limitations of the current approach, and what could remain to be done. As an example, I guess from the paper that the authors' approach mostly relies on the assumption that only a few (legitimate) entities are able to build and train state-of-the-art GANs. I may be somewhat speculative due to my lack of expertise in GANs, but I guess that as long as the cost for building such models will decrease, more malicious entities will be able to build their models, thereby mitigating the impact of fingerprinting. It might be valuable to add a short comment in this sense, even if this does not question the good quality of this submission. 

Likewise, the model inventor is assumed to be honest here. This naturally goes beyond the scope of this paper, but maybe discussing further works about non-repudiation could represent a nice open question towards the responsible release of industry-wide models.

A few minor remarks can be found hereafter. 

**Sec 3.1**
* The parameters $\lambda_i$ are arbitrarily fixed to similar values in order of magnitude, which is not discussed. While I guess that this is beyond the scope of this submission, is there any reason why all the $\lambda_i$ coefficient of the final objective loss should have the same order of magnitude? 

**Sec 4.1** 
* "Our method results in negligibly < 2.93 FID degrading. This is a reasonable trade-off between effectiveness and fidelity." -> could you elaborate a bit more ? More precisely, are there any reference values for FID with which the non-advised reader may compare ? 

**Sec 4.2** 
* "on one hand" -> "on the one hand" 
* $2^{128 \times 0.991} \approx 10^{38}$ instead of 36 (even better). 

**Sec 4.4** 
* The description of experiments for fingerprint presence/value attacks are clear, but remain very high-level. I have not found any detail about the implementation and training of the CNN binary classifier, nor any raw performance data. Accordingly, it does not enable us to assess whether enough efforts have been done to reach better accuracy, which may invalidate the secrecy claims. Could you elaborate a bit more on this ? 

## Addendum: major overlaps between the Related Works sections of this submission and (Yu et al., 2021)
Since this submission compares to (Yu et al., 2021) a lot, I quickly read this reference and I have found some major overlaps in the related work section (up to minor updates). To the best of my knowledge, this only concerns the related works section. Nevertheless, this may still technically represent some plagiarism. Hence, **I urge the authors to rewrite this section, or at the very least to mention that the related works section is mostly taken from (Yu et al., 2021)**. 

summary_of_the_review: This paper addresses an interesting problem about fingerprinting data generated by GANs, in order to be able to trace any misuse of deepfake. The paper improves state of the art by addressing a major shortcoming of previous works. Overall, the presented solution represents a clear impact on industry-deployed GANs.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors address the problem of fingerprinting GAN (StyleGAN2 in the proposed setting) in a scalable way, e.g. after initial processing, arbitrary fingerprints can be added. In doing so, the authors proposed to leverage modulated convolution layers which scales the output of GAN’s intermedia layers by projection embedded in a channel wise fashion. The authors also provide a set of metrics to evaluate GAN fingerprinting techniques, and accordingly demonstrate the superiority of the proposed method.
main_review: # Summary
In this paper, the authors address the problem of fingerprinting GAN (StyleGAN2 in the proposed setting) in a scalable way, e.g. after initial processing, arbitrary fingerprints can be added. In doing so, the authors proposed to leverage modulated convolution layers which scales the output of GAN’s intermedia layers by projection embedded in a channel wise fashion. The authors also provide a set of metrics to evaluate GAN fingerprinting techniques, and accordingly demonstrate the superiority of the proposed method.

# Strong Points

- This paper is well motivated to address the important problem of fingerprinting generative models that are becoming increasingly powerful recently such that provdenting, or at least identifying, its ill-use.
- Overall, this paper is well-structured and easy to follow.
- This paper presents a scalable way of fingerprinting GAN, where after one pass of training, a lot of fingerprints can be added requiring negligible additional resources.
- This paper presents a set of metrics for evaluating the tasks of GAN fingerprinting, which is important as such tasks are still new. Using these metrics, experiments show the advantage of the proposed method.

# Weak Points
- There is no visualization showing the effect of channel-wise modulation as a fingerprinting technique.. It would be helpful for readers to understand such channel-wise effects using visualization, for example like that in [1].
- The experiments are only done with resolutions 128x128 and 256x256. As modern GANs can generate much higher resolution (say, 1024x1024) with fine details, knowing the proposed method’s performance in higher resolution can help justify its motivation dealing with modern GANs.
- In Eq (5) there are four coefficients “ to balance the magnitude of each loss term.“ but how  are they chosen and what is the impact of such a choice are not elaborate? It would be helpful to provide analysis to the choice, as the readers may want to apply the proposed technique to other GAN models which likely have different loss terms and thus magnitudes. 

# Questions to Authors
Although this work as presented is interesting, there are some questions I have from reading it thoroughly. They are:
- In Page 3, the author claims that “they can barely sustain a long time against the adversarial iterations of GAN techniques.” Is there any concrete example of fingerprinting tech **not** withstanding adversarial iteration of GANs?
- It seems that the loss term $L_z$ in Eq (2) is orthogonal to other loss terms and is less connected to them. What’s the motivation behind using it besides it’s simply a nice to have from other recent works? It is a must-to-have ingredient in the whole pipeline?


# Other Comments
- As $L_z$ and $L_c$ in Eq (2) and Eq (3) respectively are calculated with the same decoder $F$, it seems to imply that they are calculated on different parts of the logits. It’s better to explicitly mention this fact to avoid confusion.

# Assessment
Overall, I think the strong points( the motivation, novel technique, and important set of metrics for such tasks that follow-up works in this direction would find useful.) are important and overweight weak points (there are more like better to know). So I would recommend a clear acceptance, although the authors are enraged to address may questions/concerns to make the draft more comprehensive.


# Ref
[1] Karras et al., 2021: alias-free generative adversarial networks


# Post-rebuttal

I would like to thank the authors for their response, in which most of my issues are solved.

summary_of_the_review: Overall, I think the strong points( the motivation, novel technique, and important set of metrics for such tasks that follow-up works in this direction would find useful.) are important and overweight weak points (there are more like better to know). So I would recommend a clear acceptance, although the authors are enraged to address may questions/concerns to make the draft more comprehensive.
review_title: null

Meta-review: The paper proposes and studies a method for the responsible disclosure of a fingerprint along with samples generated by a generative model, which has important applications in identifying "deep fakes". The authors establish both the detectability of their fingerprint-without significant loss of fidelity-as well as the robustness to perturbations. The reviewers found the problem and contributions to be important and significant, well substantiated by an extensive experimental study.
Decision: Accept (Spotlight)