Paper: A Linearly Convergent GAN Inversion-based Algorithm for Reverse Engineering of Deceptions (ICLR-2024)

Reviewer comment (Theory):
One important contribution of this paper is that it has theoretical proof. However, the proof is based on the assumption that the activation functions in the described network are smooth and twice differentiable. However, it is not a very practical assumption.

Author response:
- First, there appears to be a confusion between the role of the assumption as a means to prove theoretical results and the practicality of the method. Our theoretical results demonstrate linear convergence guarantees under our set of assumptions. We note that the only prior works on this problem (as cited in our paper), provide convergence guarantees in the setting where the weights of the GAN are random or close to random. In contrast, our work is the first to move beyond this set of assumptions and provide a different convergence analysis. While in many cases assumptions can be too strong, one can often run the associated algorithm with or without these assumptions. In other words, theoretical assumptions need not impact the practicality of the method. Moreover, in this particular case, our assumption is weaker than existing assumptions made by relevant state-of-the-art work. 

- The “Remark on Assumptions” in Section 5.2 and Appendix Section B.4 explicitly comment on the role of Assumption 1 on practical experiments and demonstrate the assumption does not affect inversion in practice. Thus, we argue our method has both theoretical and practical merits.

Reviewer comment (Experiments):
From the dataset aspect, the paper only shows results on simple datasets. There is no result validating the performance on a more complicated and widely adopted dataset such as ImageNet. The comparison baselines are also limited. The authors only compare with SBSAD. The authors mentioned other RED methods without theoretical guarantees such as [10] [11] [25], but no comparisons are included. Furthermore, the paper only has results on PGD attack. It's not sure if the method can be generalized to other more recent attack types such as AutoAttack. It's also meaningful to consider the adaptive attack case.

Author response:
- We understand the concern the reviewer has on the datasets chosen for this paper. We would like to emphasize that our work is the first one to propose not only a nonlinear framework for this problem, but also a method with theoretical guarantees (as compared to existing black-box methods for related RED problems). Even the three nonlinear evaluation datasets in the paper highlight that prior state-of-the-art attack classification results with theoretical guarantees from Thaker et al fail at modelling the clean signal. Further, the key strength of our model is that any generative model can be used to model the clean data as long as it has favorable inversion properties. The StyleGAN-XL paper provides impressive empirical GAN inversion results on Imagenet, and it can easily be substituted as the generative model in our approach. For computational reasons, we could not perform comprehensive Imagenet extensions for the rebuttal period. However, we have some preliminary results on TinyImagenet, which show that for 100 test examples using $\ell_\infty$ and $\ell_2$PGD attacks separately on an AlexNet architecture, we have already observed a signal classification accuracy increase from 11% undefended to 39% using GAN inversion. We believe that the existing theoretical results and the current set of experiments nonetheless provide strong evidence for the merits of the proposed novel approach in reverse engineering attacks. 

- We are not sure what references 10, 11, 25 in this paper are as the references are by author name.  

- Our aim is provable attack classification using a dictionary model for the attacks. The work of Thaker et al provides a theoretical justification for using the dictionary model for PGD $\ell_p$ attacks (namely that attacks on test examples can be represented as linear combinations of attacks on training examples for a network that uses ReLU activations). It is not clear whether this model makes sense for more complex attacks such as AutoAttack, and so we defer a model for these classes of attacks to future work.

Reviewer comment (Experiments):
Can this method generalize to more complicated dataset such as ImageNet?

Author response:
- We understand the concern the reviewer has on the datasets chosen for this paper. We would like to emphasize that our work is the first one to propose not only a nonlinear framework for this problem, but also a method with theoretical guarantees (as compared to existing black-box methods for related RED problems). Even the three nonlinear evaluation datasets in the paper highlight that prior state-of-the-art attack classification results with theoretical guarantees from Thaker et al fail at modelling the clean signal. Further, the key strength of our model is that any generative model can be used to model the clean data as long as it has favorable inversion properties. The StyleGAN-XL paper provides impressive empirical GAN inversion results on Imagenet, and it can easily be substituted as the generative model in our approach. For computational reasons, we could not perform comprehensive Imagenet extensions for the rebuttal period. However, we have some preliminary results on TinyImagenet, which show that for 100 test examples using $\ell_\infty$ and $\ell_2$PGD attacks separately on an AlexNet architecture, we have already observed a signal classification accuracy increase from 11% undefended to 39% using GAN inversion. We believe that the existing theoretical results and the current set of experiments nonetheless provide strong evidence for the merits of the proposed novel approach in reverse engineering attacks.

Reviewer comment (Evaluation):
How is the recovered clean signal look like? Is it close to the groundtruth clean example?

Author response:
Appendix Section B.4 provides qualitative results for reconstruction error and we observe that the reconstructed image is indeed close to the ground truth denoised image.

Reviewer comment (Presentation):
In the introduction section of this article, authors use too much space to introducing the background, and takes too long to present the problem. The article lacks effective organization.

Author response:
Thank you for the feedback. We have condensed the introduction slightly to motivate the problem setup of RED more directly and uploaded a revision. We felt it was important to motivate the problem setup of RED in this way since it is distinctly different from standard literature in adversarial learning. Further, our approach builds closely on the prior work of Thaker et al, so we believe the discussion of the flaws in this work is crucial to understanding our contributions. We hope the reviewer finds that the current modifications to the introduction address their comments, but we would appreciate if the reviewer has more specific feedback on the presentation. Moreover, the rest of the paper is structured in a didactic manner: starting from problem setup, moving to a theoretical analysis structured in 3 settings that go in order of added complexity, and concluding with a discussion of experimental results. We respectfully disagree that the article lacks effective organization, but we would appreciate any specific feedback the reviewer has on improving readability and organization.

Reviewer comment (Writing):
The presentation of this article needs improvement, as there are many grammar issues, and it lacks readability. For example, in the third sentence of second paragraph, citing the related works between two commas.

Author response:
We respectfully disagree with the reviewer that there are many grammar issues with the article. Citing related works between two commas seems like a style choice as opposed to a grammar issue. To be consistent though, we have removed the extra comma in the three other citations in the paper which had that issue. Upon a thorough proofread, we have found no other grammar issues. We would appreciate if the reviewer has specific suggestions to further improve readability.

Reviewer comment (Evaluation):
The results of the experiment cannot strongly support the algorithm's advantages.

Author response:
We attempted to perform experiments that convincingly showcase our theoretical results and the merits of the proposed algorithm, and we are glad that the reviewer found that we “meticulously designed experiments, thoroughly analyzed the experimental results, and demonstrated the effectiveness of the algorithm.” We understand the concern the reviewer has on the datasets chosen for this paper. We would like to emphasize that our work is the first one to propose not only a nonlinear framework for this problem, but also a method with theoretical guarantees (as compared to existing black-box methods for related RED problems). Even the three nonlinear evaluation datasets in the paper highlight that prior state-of-the-art attack classification results with theoretical guarantees from Thaker et al fail at modelling the clean signal. Further, the key strength of our model is that any generative model can be used to model the clean data as long as it has favorable inversion properties. The StyleGAN-XL paper provides impressive empirical GAN inversion results on Imagenet, and it can easily be substituted as the generative model in our approach. For computational reasons, we could not perform comprehensive Imagenet extensions for the rebuttal period. However, we have some preliminary results on TinyImagenet, which show that for 100 test examples using $\ell_\infty$ and $\ell_2$ PGD attacks separately on an AlexNet architecture, we have already observed a signal classification accuracy increase from 11% undefended to 39% using GAN inversion. We believe that the existing theoretical results and the current set of experiments nonetheless provide strong evidence for the merits of the proposed novel approach in reverse engineering attacks.

Reviewer comment (Writing):
My mainly concern is with the writing of the article. The sentences are not easy to understand, there are grammar errors, and the readability is poor.

Author response:
We respectfully disagree with the reviewer that there are many grammar issues with the article. Citing related works between two commas seems like a style choice as opposed to a grammar issue. To be consistent though, we have removed the extra comma in the three other citations in the paper which had that issue. Upon a thorough proofread, we have found no other grammar issues. We would appreciate if the reviewer has specific suggestions to further improve readability.

Reviewer comment (Experiments):
The paper could enhance its comparison with State-of-the-Art methods, particularly by evaluating signal accuracy in comparison to other leading works. A broader array of these methods should be taken into account for a more comprehensive comparison.

Author response:
Thank you for the feedback. In the revised Appendix of the paper, we have added section B.8 and Table 6 which provide a comparison of our approach and the method that uses diffusion models for the underlying generative model instead of GANs. We show the table here as well. Using our approach, one can alternate between denoising using diffusion models and classifying the attack using the attack dictionary as we have done in this work. Specifically, let $D(x')$ be the denoised version of $x'$ by passing through the forward and reverse process of a diffusion model $D$ (DiffPure approach from Nie et al.) . Then, our modelling assumption is that $x' \approx D(x') + D_a c_a$. Our alternating algorithm then becomes in each step to perform two operations: a)  compute $D(x')$ b) use proximal gradient descent steps to fit the model for $c_a$. We call this method BSD-DP and BSD-DP-AD for signal and attack classification respectively (where DP stands for DiffPure). 

We find that while the diffusion model is a powerful denoiser and yields slightly improved signal classification accuracy on average over different $\ell_p$ attacks, it does not result in accurate reconstructions of the original image, which yields poor attack classification accuracy compared to our approach using GANs. Please refer to Appendix Section B.8 for more analysis. Further, a strong benefit of our approach is that it comes with theoretical guarantees on GAN inversion and attack classification, whereas provable denoising using diffusion models with complex nonlinear reverse processes is an open problem.  

### Adversarial image and attack classification accuracy on CIFAR-10 dataset for 100 test examples. See text above for column descriptions. DP stands for DiffPure.

| CIFAR-10 | CNN | BSD-GAN | DP | BSD-DP | BSD-GAN-AD | BSD-DP-AD |
|----------|-----|---------|----|--------|------------|------------|
| Clean accuracy | 99% | 72% | 84% | 84% | - | - |
| $\ell_\infty$ PGD ($\epsilon = 0.03$) | 0% | 76% | 82% | **87%** | **48%** | 0% |
| $\ell_2$ PGD ($\epsilon = 0.5$) | 0% | **87%** | 83% | **87%** | 77% | **100%** |
| $\ell_1$ PGD ($\epsilon = 12.0$) | 0% | 71% | 82% | **87%** | 44% | 0% |
| Average | 0% | 78% | 82.33% | **87%** | **56%** | 33.33% |

Reviewer comment (Experiments):
The assessment could extend to more sophisticated datasets, like ImageNet, as numerous other State-of-the-Art methods have previously reported their performance on such datasets.

Author response:
We understand the concern the reviewer has on the datasets chosen for this paper. We would like to emphasize that our work is the first one to propose not only a nonlinear framework for this problem, but also a method with theoretical guarantees (as compared to existing black-box methods for related RED problems). Even the three nonlinear evaluation datasets in the paper highlight that prior state-of-the-art attack classification results with theoretical guarantees from Thaker et al fail at modelling the clean signal. Further, the key strength of our model is that any generative model can be used to model the clean data as long as it has favorable inversion properties. The StyleGAN-XL paper provides impressive empirical GAN inversion results on Imagenet, and it can easily be substituted as the generative model in our approach. For computational reasons, we could not perform comprehensive Imagenet extensions for the rebuttal period. However, we have some preliminary results on TinyImagenet, which show that for 100 test examples using $\ell_\infty$ and $\ell_2$ PGD attacks separately on an AlexNet architecture, we have already observed a signal classification accuracy increase from 11% undefended to 39% using GAN inversion. We believe that the existing theoretical results and the current set of experiments nonetheless provide strong evidence for the merits of the proposed novel approach in reverse engineering attacks.

Reviewer comment (Experiments):
To empirically demonstrate the Error Bound, it would be advantageous to present it across a set of samples with varying sizes and illustrate its consistency. If the consistency holds, the maximum/standard deviation should remain nearly constant with an increase in the number of data samples.

Author response:
While we are not exactly sure what the reviewer means by consistency here, we interpret it as varying the number of samples over which the local error bound parameter is calculated to see if the trend holds more broadly. Note that the GAN inversion is ran as a different optimization problem per datapoint, and the landscape shown in Figure 2 holds for all choices of $z^*$ (as Corollary 4 argues), even though only one such figure is shown. For deep networks with random weights, following the setup in Appendix Section B.3, we have rerun the local error bound calculation with a varying number of datapoints over which the average $\mu$ is computed. We found that the average and standard deviation is roughly constant in the increase of data samples as expected. We thank you for the suggestion and hope this is a satisfactory answer to your point.  

### Consistency of Local Error Bound Condition as Number of Samples Increases 
 | # Examples | 10     | 25     | 50     | 75     | 100    | 150    | 200    |
|------------|--------|--------|--------|--------|--------|--------|--------|
| Average    | 470.74 | 487.99 | 434.32 | 502.43 | 508.65 | 468.71 | 447.82 |
| Standard deviation | 268.45 | 218.97 | 192.40 | 199.34 | 254.58 | 236.02 | 310.11 |

Reviewer comment (Experiments):
The effectiveness of GAN inversion falls short compared to the performance achieved with diffusion models, which are commonly employed in current practical applications. Some studies, such as [Nie et al.](https://arxiv.org/abs/2205.07460), have explored the use of Diffusion Models for purification, resulting in enhanced performance. It might be worthwhile to incorporate these models in this context, considering their potential for estimating clean data. Additionally, comparing the results against such approaches could provide valuable insights.

Author response:
Thank you for the feedback. In the revised Appendix of the paper, we have added section B.8 and Table 6 which provide a comparison of our approach and the method that uses diffusion models for the underlying generative model instead of GANs. We show the table here as well. Using our approach, one can alternate between denoising using diffusion models and classifying the attack using the attack dictionary as we have done in this work. Specifically, let $D(x')$ be the denoised version of $x'$ by passing through the forward and reverse process of a diffusion model $D$ (DiffPure approach from Nie et al.) . Then, our modelling assumption is that $x' \approx D(x') + D_a c_a$. Our alternating algorithm then becomes in each step to perform two operations: a)  compute $D(x')$ b) use proximal gradient descent steps to fit the model for $c_a$. We call this method BSD-DP and BSD-DP-AD for signal and attack classification respectively (where DP stands for DiffPure). 

We find that while the diffusion model is a powerful denoiser and yields slightly improved signal classification accuracy on average over different $\ell_p$ attacks, it does not result in accurate reconstructions of the original image, which yields poor attack classification accuracy compared to our approach using GANs. Please refer to Appendix Section B.8 for more analysis. Further, a strong benefit of our approach is that it comes with theoretical guarantees on GAN inversion and attack classification, whereas provable denoising using diffusion models with complex nonlinear reverse processes is an open problem.  

### Adversarial image and attack classification accuracy on CIFAR-10 dataset for 100 test examples. See text above for column descriptions. DP stands for DiffPure.

| CIFAR-10 | CNN | BSD-GAN | DP | BSD-DP | BSD-GAN-AD | BSD-DP-AD |
|----------|-----|---------|----|--------|------------|------------|
| Clean accuracy | 99% | 72% | 84% | 84% | - | - |
| $\ell_\infty$ PGD ($\epsilon = 0.03$) | 0% | 76% | 82% | **87%** | **48%** | 0% |
| $\ell_2$ PGD ($\epsilon = 0.5$) | 0% | **87%** | 83% | **87%** | 77% | **100%** |
| $\ell_1$ PGD ($\epsilon = 12.0$) | 0% | 71% | 82% | **87%** | 44% | 0% |
| Average | 0% | 78% | 82.33% | **87%** | **56%** | 33.33% |

Reviewer comment (Experiments):
In practical applications, subgradient descent is typically not employed, as was done in your optimization for GAN inversion. Have you compared your GAN inversion results against those obtained using alternative methods?

Author response:
We designed most of our experiments to be in the regime that our theoretical results were applicable in and comparable to existing theoretical results in GAN inversion such as the work of Hand et al. As our convergence theory is proven for subgradient descent, we chose that algorithm for all our experiments. However, even in practical applications, we have seen inversion using subgradient descent (for example from StyleGAN-XL paper) but in the style space of the GAN, which is what we adopted for our StyleGAN experiments as well. We note though that the proposed framework is general and any GAN inversion algorithm could be used in conjunction with proximal gradient descent.