Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors consider the problem of model inversion using an auxiliary dataset identically distributed to the training data, its predictions under the model as well as their explanation maps under Grad, Grad-Cam, and LRP saliency map. Given a trained target classifier, their approach consists in setting up an adversarial game between a noise injection and model inversion networks. The noise injection network returns a censored version of the explanation by minimizing the euclidean distance between the explanation and its censored version while maximizing the euclidean distance between the original image and its reconstruction by the inversion model network. The inversion model network aims to predict the original images from the target model predictions, explanations and the injection network censored explanation.
strengths_and_weaknesses: Strengths:

* The paper is clearly written and easy to follow.
* The authors consider a wide range of baselines and quantitative evaluation metric.
* The author quantitatively show that censored explanations remain interpretable.

Weaknesses:
* The paper only consider small scale image data. 
* It's not clear whether the algorithm converge or not. 
questions: The paper would be substantially strengthened and find a wider audience by considering tabular and text data. 
Considering large scale, complex image data would give an opportunity for the method to set itself apart even more. 
A theoretical analysis of the algorithm convergence, even in a non-parametric setting and just at optimality, could shed light on the relationship between the noise injection and model inversion network after training. 
limitations: Yes.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a perturbation-based approach to prevent an adversary from leveraging model explanations to perform model inversion attacks. In particular, the authors apply this defence mechanism to three explanation types: Grad, Grad-CAM, and LRP. The proposed adversarial training framework consists of a model inversion network that performs the inversion attack and a generator network that learn to create perturbations for explanations. Experiments are performed to demonstrate the proposed approach's ability to reduce the efficacy of explanation-aware model inversion attacks.
strengths_and_weaknesses: **Strengths** 

-   Several evaluation metrics to compare the original input $x$ with the reconstructed input $\hat{x}$, namely MSE, SSIM, DeePSIM, and TCA
- The proposed defence mechanism is evaluated against state-of-the-art prediction-based and explanation-aware model inversion attacks
- Experiments are performed on several real-world datasets
- Realistic assumptions on the adversary knowledge have been investigated

**Weaknesses** 

My main concern is the lack of discussion on the manipulability of saliency maps. Several research works have shown (e.g., [a]) that these types of explanation techniques can be arbitrarily manipulated. While the paper is not concerned with adversarial manipulation of explanations, the noise added to the original explanations can also unintentionally leads to manipulated explanations that fairwashed the original decisions of the target model. 

[a] - Explanations can be manipulated and geometry is to blame. (Dombrowski et al., 2019)
questions: - Please add more discussion on the risk that noisy explanations can lead to manipulated explanations. 
- If possible, report the additional score like the SSIM of $x \odot e$ and  $x \odot \hat{e}$ to provide more arguments for the preservation of XAI functionality.
limitations: Nothing to report.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposes a defense method that add noise to model explanations to minimize the risk of model inversion attacks, and preserving the interpretabilities of the explanations. They formulate the problem of defense training as a two-player minimax game between the MI attacker and the noise generator.  The MI attacker takes in the explanation and prediction vector, generates the original input, and the noise generator takes in the model explanation and prediction vector and generates a perturbed version of the explanation in order to prevent the MI attacker from leaking information. The proposed framework is able to defense against MI attack according to the results, achieving a larger reconstruction error than other baselines, and it is able to preserve the effectiveness of the model explanations to some extent. 
strengths_and_weaknesses: originality: the proposal to formulate the defense problem as a min-max game is very interesting and the results show that it works to some extent. The proposed evaluation metrics clearly show the advantage of the proposed method over the baselines such as gaussian noise injection.

quality and clarity: the work's method is clear and experiments include detailed comparisons with several baselines. However, with so many detailed quantitative results, it is not clear whether the achieved results are significant or not. Maybe there could be a better way to present the data so that it is not so messy and could be clearer for readers to understand the results. Plots could be a better option than tables to show the results since they are more straightforward and intuitive. Many experimental details are included in the text, which makes it hard to understand the big picture. Maybe a better way is to have a table summarizing the details of various hyper parameters, and have more subsections each covering one topic. 

Significance: the method itself is interesting and valuable to the community as a way to defend against MI attack. However, it is less clear to me how significant are the results. On one hand, the used dataset is relatively small, what about the results on large scale ImageNet dataset? On the other hand, the reconstructed image even after the noise perturbation on the explanation can still reveal some characteristics of the original image. For example, the gender and some facial features on the human face images can be identified, even if the proposed method achieves better metrics than baselines. 



 
questions: From the results of the paper, the proposed method achieves better performance than the mentioned baselines. However, the reconstructed image can still reveal some features of the original image. Is there a potential risk that even the revealed partial information may be abused by attackers to reveal privacy of the original image?

The results provide detailed numbers of performance metrics, but it is not clear whether the improvements are significant or not. Is there a better way to show that the improvements are statistically significant? 
limitations: To some extent, but the authors did not mention what other information could be leaked even given the blurry reconstructed images. Ideally, the reconstructed image should be very blurred and cannot be recognized as human face.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors proposed a defense methodology against model inversion attack, which uses model explanation results to recover the training set. The purposes of this defense method are to minimize the attack success rate and keep the model's interpretability. The authors train noise generator and inversion network to achieve these purposes by using a minimax setting. They evaluated the efficacy of their methodology by measuring defense success rate and interpretability using various datasets.

strengths_and_weaknesses: Strength
1. Proposed first defense method against explanation aware model inversion attack.
2. Did the same experiment several times to show the consistency of their method.

Weakness
1. The image data used were limited to low resolution. Since the methodology preserves the privacy of the training set, higher resolution images of human figures or X-rays would be a good choice to check the efficacy of methodology. 
questions: 1. Is there a threat when the adversary knows the fact that the GNIME is used? 
limitations: Limitation is enoughly discussed in limitation section of main paper.

<minor suggestion>

1. In line 286, please specify the table.
2. Please add the attribution maps for qualitative comparison. (Fig 5. is too small to see the difference.)


---
Thank you for the additional experiments and explanations.

The additional experiments show that GNIME is also working for the higher resolution images from Imagenet(256x256).
However, I expected 512 or 1024 resolution of human figures (e.g., FFHQ) or X-ray. 
These datasets are more important for privacy. 

The explanation for other adversary scenario is great.
But I am going to maintain the score since the additional experiments are not resolving weakness.
review_title: null

Meta-review: metareview: This paper proposes the GNIME method that defends against explanation-aware model inversion attacks by adversarially manipulating the saliency map. The reviews are mostly positive. The (minor) concerns and suggestions are

- The paper lacks a theoretical analysis of the adversarial framework. A theoretical analysis even at optimality will improve the paper. However, this is not a major concern as the experiment results are quite good.

- The idea of adversarially manipulating a representation (here a saliency map) so as to remove sensitive factors of variation while preserving as much information as possible, is not new. For example, these adversarial frameworks have been studied in the context for learning fair representations, and in many other areas. I suggest including a discussion about this in the final manuscript.

confidence: Certain
Decision: Accept