Original: Learning to Generate Inversion-Resistant Model Explanations
Revised: Learning to Generate Inversion-Resistant Model Explanations

Original: Abstract
Revised: Abstract

Original: The wide adoption of deep neural networks (DNNs) in mission-critical applications has spurred the need for interpretable models that provide explanations of the model’s decisions.
Revised: The wide adoption of deep neural networks (DNNs) in mission-critical applications has spurred the need for interpretable models that provide explanations of the model’s decisions.

Original: Unfortunately, previous studies have demonstrated that model explanations facilitate information leakage, rendering DNN models vulnerable to model inversion attacks.
Revised: Unfortunately, previous studies have demonstrated that model explanations facilitate information leakage, rendering DNN models vulnerable to model inversion attacks.

Original: These attacks enable the adversary to reconstruct original images based on model explanations, thus leaking privacy-sensitive features.
Revised: These attacks enable the adversary to reconstruct original images based on model explanations, thus leaking privacy-sensitive features.

Original: To this end, we present Generative Noise Injector for Model Explanations (GNIME), a novel defense framework that perturbs model explanations to minimize the risk of model inversion attacks while preserving the interpretabilities of the generated explanations.
Revised: To this end, we present Generative Noise Injector for Model Explanations (GNIME), a novel defense framework that perturbs model explanations to minimize the risk of model inversion attacks while preserving the interpretabilities of the generated explanations.

Original: Specifically, we formulate the defense training as a two-player minimax game between the inversion attack network on the one hand, which aims to invert model explanations, and the noise generator network on the other, which aims to inject perturbations to tamper with model inversion attacks.
Revised: Specifically, we formulate the defense training as a two-player minimax game between the inversion attack network on the one hand, which aims to invert model explanations, and the noise generator network on the other, which aims to inject perturbations to tamper with model inversion attacks.

Original: We demonstrate that GNIME significantly decreases the information leakage in model explanations, decreasing transferable classification accuracy in facial recognition models by up to 84.8% while preserving the original functionality of model explanations.
Revised: We demonstrate that GNIME significantly decreases the information leakage in model explanations, decreasing transferable classification accuracy in facial recognition models by up to 84.8% while preserving the original functionality of model explanations.

Original:  Introduction
Revised:  Introduction

Original: The recent widespread adoption of deep neural networks (DNNs) in building practical real-world applications has led to a surge of interest in explainable AI (XAI) since mission-critical tasks (e g., medical diagnosis) often request explanations of the model’s predictions.
Revised: The recent widespread adoption of deep neural networks (DNNs) in building practical real-world applications has led to a surge of interest in explainable AI (XAI) since mission-critical tasks (e g., medical diagnosis) often request explanations of the model’s predictions.

Original: Such explanations are not only helpful in debugging the model but also in providing explanations to end-users on how predictions are made.
Revised: Such explanations are not only helpful in debugging the model but also in providing explanations to end-users on how predictions are made.

Original: They are often provided in the form of a feature attribution map which shows the importance of each feature computed based on the gradients of a target model.
Revised: They are often provided in the form of a feature attribution map which shows the importance of each feature computed based on the gradients of a target model.

Original: Unfortunately, recent studies have demonstrated that such explanations cause additional information leakage, which can be exploited by an adversary [1, 8, 19, 22, 32].
Revised: Unfortunately, recent studies have demonstrated that such explanations cause additional information leakage, which can be exploited by an adversary [1, 8, 19, 22, 32].

Original: Specifically, Zhao et al.
Revised: Specifically, Zhao et al.

Original: [32] presented a novel black-box model inversion (MI) attack which reconstructs an input image given its prediction vector and the model explanation from a target DNN model.
Revised: [32] presented a novel black-box model inversion (MI) attack which reconstructs an input image given its prediction vector and the model explanation from a target DNN model.

Original: They demonstrated that additional information leaked through model explanations (e.g., Grad or Grad-CAM) increases the performance of MI attacks by a factor of 2.4 in terms of transferable classification accuracy, enabling high-fidelity attacks.
Revised: They demonstrated that additional information leaked through model explanations (e.g., Grad or Grad-CAM) increases the performance of MI attacks by a factor of 2.4 in terms of transferable classification accuracy, enabling high-fidelity attacks.

Original: Figure 1 compares the result of MI attacks using model predictions alone (PredMI) and explanationaware MI attacks using model predictions and explanations together (ExpMI).
Revised: Figure 1 compares the result of MI attacks using model predictions alone (PredMI) and explanationaware MI attacks using model predictions and explanations together (ExpMI).

Original: The examples reconstructed via ExpMI show high-quality faces compared to those via PredMI, leaking additional information, such as facial expressions and accessories.
Revised: The examples reconstructed via ExpMI show high-quality faces compared to those via PredMI, leaking additional information, such as facial expressions and accessories.

Original: However, previous works have only proposed the attacks exploiting the information leakage, without providing any defenses against them.
Revised: However, previous works have only proposed the attacks exploiting the information leakage, without providing any defenses against them.

Original: Thus, to defend against MI attacks exploiting model explanations, we propose GNIME, which generates inversion-resistant model explanations.
Revised: Thus, to defend against MI attacks exploiting model explanations, we propose GNIME, which generates inversion-resistant model explanations.

Original: Figure 2 shows the overall GNIME pipeline.
Revised: Figure 2 shows the overall GNIME pipeline.

Original: GNIME consists of two phases: training and deployment.
Revised: GNIME consists of two phases: training and deployment.

Original: In the training phase, GNIME trains a noise generator that injects perturbations into the explanations provided by the model under protection to minimize the MI threat.
Revised: In the training phase, GNIME trains a noise generator that injects perturbations into the explanations provided by the model under protection to minimize the MI threat.

Original: To train this noise generator, GNIME trains an MI attack network to compete against each other.
Revised: To train this noise generator, GNIME trains an MI attack network to compete against each other.

Original: Then, in the deployment phase, we deploy the trained noise generator to obfuscate the explanations generated for each test instance.
Revised: Then, in the deployment phase, we deploy the trained noise generator to obfuscate the explanations generated for each test instance.

Original: [Figure]
Revised: [Figure]

Original: The key idea of our approach is to establish a two-player minimax game between the MI attack (INV) network and the noise generator (NG) network.
Revised: The key idea of our approach is to establish a two-player minimax game between the MI attack (INV) network and the noise generator (NG) network.

Original: We design the NG network to inject perturbations into the model’s explanations (i.e., attribution map); the noise-injected model explanation is then fed into the INV network to reconstruct the inverted image.
Revised: We design the NG network to inject perturbations into the model’s explanations (i.e., attribution map); the noise-injected model explanation is then fed into the INV network to reconstruct the inverted image.

Original: GNIME conducts an alternating training of these networks.
Revised: GNIME conducts an alternating training of these networks.

Original: During the NG’s turn, the NG network is trained to tamper with the inversion performance of the INV network while minimizing the perturbations to the model explanations.
Revised: During the NG’s turn, the NG network is trained to tamper with the inversion performance of the INV network while minimizing the perturbations to the model explanations.

Original: During the
Revised: During the

Original: INV’s turn, we train the INV network to maximize its model inversion performance.
Revised: INV’s turn, we train the INV network to maximize its model inversion performance.

Original: These alternating training procedures improve the INV network to better invert the given model’s explanations while the NG network learns to inject perceptually imperceptible inversion-resistant perturbations to the provided explanations.
Revised: These alternating training procedures improve the INV network to better invert the given model’s explanations while the NG network learns to inject perceptually imperceptible inversion-resistant perturbations to the provided explanations.

Original: After training is complete, the GNIME owner deploys the NG network to inject perturbations into the original explanations before releasing them.
Revised: After training is complete, the GNIME owner deploys the NG network to inject perturbations into the original explanations before releasing them.

Original: We evaluate the efficacy of GNIME, by measuring 1) how effective the model is in decreasing the risk of information leakage from the model explanations and 2) how well the perturbed explanations preserve the semantics of the original explanations.
Revised: We evaluate the efficacy of GNIME, by measuring 1) how effective the model is in decreasing the risk of information leakage from the model explanations and 2) how well the perturbed explanations preserve the semantics of the original explanations.

Original: For different types of explanations (i.e., Grad [24],
Revised: For different types of explanations (i.e., Grad [24],

Original: Grad-CAM [21], and LRP [3]), we demonstrate that GNIME consistently generates infinitesimal inversion-resistant noises that significantly impede explanation-aware MI attacks.
Revised: Grad-CAM [21], and LRP [3]), we demonstrate that GNIME consistently generates infinitesimal inversion-resistant noises that significantly impede explanation-aware MI attacks.

Original: To facilitate further research, we publish GNIME at https://github.com/WSP-LAB/GNIME .
Revised: To facilitate further research, we publish GNIME at https://github.com/WSP-LAB/GNIME .

Original: In summary, our contributions are as follows:
Revised: In summary, our contributions are as follows:

Original: • We propose the very first defense framework against explanation-aware MI attacks, which learns to suppress inversion-critical features from model explanations.
Revised: • We propose the very first defense framework against explanation-aware MI attacks, which learns to suppress inversion-critical features from model explanations.

Original: • We evaluate GNIME on diverse datasets and demonstrate its efficacy in decreasing the MI risk while preserving the interpretability of the original explanations.
Revised: • We evaluate GNIME on diverse datasets and demonstrate its efficacy in decreasing the MI risk while preserving the interpretability of the original explanations.

Original:  Related Work
Revised:  Related Work

Original: Explainable AI for CNN models.
Revised: Explainable AI for CNN models.

Original: Previous XAI research has introduced diverse methods of computing an attribution map (i.e., saliency map) that explains the decision of a DNN classifier.
Revised: Previous XAI research has introduced diverse methods of computing an attribution map (i.e., saliency map) that explains the decision of a DNN classifier.

Original: Specifically, given an input image, its attribution map highlights important features that heavily influence the classifier’s decision.
Revised: Specifically, given an input image, its attribution map highlights important features that heavily influence the classifier’s decision.

Original: A straightforward approach is to calculate the gradient of the
Revised: A straightforward approach is to calculate the gradient of the

Original: DNN model’s output with respect to each input pixel [24], which we refer to as Grad.
Revised: DNN model’s output with respect to each input pixel [24], which we refer to as Grad.

Original: Grad thus produces an attribution map of which the size is identical to that of input images.
Revised: Grad thus produces an attribution map of which the size is identical to that of input images.

Original: The larger the absolute Grad value of a pixel, the greater influence the corresponding pixel has in deriving a model’s decision.
Revised: The larger the absolute Grad value of a pixel, the greater influence the corresponding pixel has in deriving a model’s decision.

Original: Numerous variants have been proposed to supplement or extend the explainability of
Revised: Numerous variants have been proposed to supplement or extend the explainability of

Original: Grad [23, 25, 26].
Revised: Grad [23, 25, 26].

Original: In this paper, we use Grad as a basic type of for model explanations.
Revised: In this paper, we use Grad as a basic type of for model explanations.

Original: The class activation map (CAM) [33] is also a prevalent means of computing model explanations.
Revised: The class activation map (CAM) [33] is also a prevalent means of computing model explanations.

Original: It provides a coarse-grained activation map that represents the state of intermediate convolutional layers.
Revised: It provides a coarse-grained activation map that represents the state of intermediate convolutional layers.

Original: Gradientweighted CAM or Grad-CAM [21] is an extension of CAM.
Revised: Gradientweighted CAM or Grad-CAM [21] is an extension of CAM.

Original: Unlike CAM, Grad-CAM does not require a specific model architecture.
Revised: Unlike CAM, Grad-CAM does not require a specific model architecture.

Original: It performs partial back-propagation until reaching the final
Revised: It performs partial back-propagation until reaching the final

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: convolutional layer before the fully-connected (FC) layers.
Revised: convolutional layer before the fully-connected (FC) layers.

Original: The activation maps are then aggregated by globally pooling all existing channels, thus constituting a heatmap, which can be overlaid upon the original image.
Revised: The activation maps are then aggregated by globally pooling all existing channels, thus constituting a heatmap, which can be overlaid upon the original image.

Original: The layer-wise relevance propagation (LRP) takes a holistic approach to explaining a model’s decision [3].
Revised: The layer-wise relevance propagation (LRP) takes a holistic approach to explaining a model’s decision [3].

Original: It performs importance decomposition by redistributing relevance scores along the backward pass and derives the relative contribution of each pixel of the input image.
Revised: It performs importance decomposition by redistributing relevance scores along the backward pass and derives the relative contribution of each pixel of the input image.

Original: Model Inversion Attacks.
Revised: Model Inversion Attacks.

Original: An MI attack refers to an attempt to invert a given model’s output, thus computing its approximate input.
Revised: An MI attack refers to an attempt to invert a given model’s output, thus computing its approximate input.

Original: The first MI attack targeted simple linear models to reconstruct certain genetic markers from medical dosage predictions [10].
Revised: The first MI attack targeted simple linear models to reconstruct certain genetic markers from medical dosage predictions [10].

Original: Fredrikson et al.
Revised: Fredrikson et al.

Original: [9] expanded this idea to target other types of machine learning (ML) algorithms, such as decision trees and shallow neural networks, which enable the attack to target facial recognition models.
Revised: [9] expanded this idea to target other types of machine learning (ML) algorithms, such as decision trees and shallow neural networks, which enable the attack to target facial recognition models.

Original: These studies leveraged the maximum a posteriori principle to construct input features that maximize the likelihood of a given model’s response.
Revised: These studies leveraged the maximum a posteriori principle to construct input features that maximize the likelihood of a given model’s response.

Original: However, their reconstructed images were blurry and of low-resolution; these methods are unfit for reconstructing input images against complicated DNN models.
Revised: However, their reconstructed images were blurry and of low-resolution; these methods are unfit for reconstructing input images against complicated DNN models.

Original: Yang et al.
Revised: Yang et al.

Original: [29] proposed the idea of utilizing a dedicated DNN model to perform MI attacks.
Revised: [29] proposed the idea of utilizing a dedicated DNN model to perform MI attacks.

Original: They pretrained this inversion model with inputs that the attacker generates and their corresponding outputs from a target DNN model.
Revised: They pretrained this inversion model with inputs that the attacker generates and their corresponding outputs from a target DNN model.

Original: This training-based MI attack drastically improved the model inversion performance.
Revised: This training-based MI attack drastically improved the model inversion performance.

Original: Follow-up works have demonstrated the use of generative adversarial networks (GANs) to improve the inversion performance [2, 5, 27, 31].
Revised: Follow-up works have demonstrated the use of generative adversarial networks (GANs) to improve the inversion performance [2, 5, 27, 31].

Original: Recently, Zhao et al.
Revised: Recently, Zhao et al.

Original: [32] showed that the additional information leakage from XAI models can be further exploited to enhance the effectiveness of the MI attack.
Revised: [32] showed that the additional information leakage from XAI models can be further exploited to enhance the effectiveness of the MI attack.

Original: While previous GAN-based attacks focus on producing generic images of a target class, the explanation-aware attack by Zhao et al.
Revised: While previous GAN-based attacks focus on producing generic images of a target class, the explanation-aware attack by Zhao et al.

Original: [32] is able to reconstruct features specific to test instances, such as face images with hats, glasses, or earrings.
Revised: [32] is able to reconstruct features specific to test instances, such as face images with hats, glasses, or earrings.

Original: However, to our knowledge, no defense mechanism has been proposed to defend against such explanation-aware MI attacks.
Revised: However, to our knowledge, no defense mechanism has been proposed to defend against such explanation-aware MI attacks.

Original:  Explanation-aware Model Inversion Attack
Revised:  Explanation-aware Model Inversion Attack

Original: Previous studies have demonstrated the critical threat that generative MI attacks impose.
Revised: Previous studies have demonstrated the critical threat that generative MI attacks impose.

Original: In this section, we describe the threat model for conducting state-of-the-art generative MI attacks against which GNIME is designed to protect model explanations.
Revised: In this section, we describe the threat model for conducting state-of-the-art generative MI attacks against which GNIME is designed to protect model explanations.

Original: We then explain the state-of-the-art explanation-aware MI attacks using generative networks along with notations.
Revised: We then explain the state-of-the-art explanation-aware MI attacks using generative networks along with notations.

Original: 3.1 Threat Model
Revised: 3.1 Threat Model

Original: We consider a target classifier f providing black-box access whereby any user can query f to obtain its output ˆ y in the form of a prediction vector.
Revised: We consider a target classifier f providing black-box access whereby an adversary can query f to obtain its output ˆ y in the form of a prediction vector.

Original: Each vector value indicates the predicted probability for the corresponding class.
Revised: Each vector value indicates the predicted probability for the corresponding class.

Original: Furthermore, the target service leveraging f also provides a model explanation e (e.g., a Grad, Grad-CAM, or LRP saliency map) that illustrates important features contributing to f emitting ˆ y .
Revised: Furthermore, the target service leveraging f also provides a model explanation e (e.g., a Grad, Grad-CAM, or LRP saliency map) that illustrates important features contributing to f emitting ˆ y .

Original: We assume an eavesdropping adversary who observes a pair of e and ˆ y for an input image inaccessible to the adversary.
Revised: We assume a man-in-the-middle adversary who observes a pair of e and ˆ y for an input image inaccessible to the adversary.

Original: Thus, the adversary’s goal is to reconstruct a high-fidelity input image given e and ˆ y .
Revised: Thus, the adversary’s goal is to reconstruct a high-fidelity input image given e and ˆ y .

Original: Furthermore, we assume that the adversary is able to obtain an auxiliary dataset D aux whose underlying data distribution is similar to that of the training data D for the target classifier f .
Revised: Furthermore, we assume that the adversary is able to obtain an auxiliary dataset D aux whose underlying data distribution is similar to that of the training data D for the target classifier f .

Original: The adversary leverages this auxiliary dataset to train her inversion networks f A .
Revised: The adversary leverages this auxiliary dataset to train her inversion networks f A .

Original: Since the adversary can collect (ˆ y, e ) by querying f for each input x ∈ D aux , she exploits both ˆ y and e to train f A .
Revised: Since the adversary can collect (ˆ y, e ) by querying f for each input x ∈ D aux , she exploits both ˆ y and e to train f A .

Original: Note that our adversary model is analogous to that of Zhao et al.
Revised: Note that our adversary model is analogous to that of Zhao et al.

Original: [32], which constitutes a strong black-box adversary model.
Revised: [32], which constitutes a strong black-box adversary model.

Original: Also, Yang et al.
Revised: Also, Yang et al.

Original: [29] assumes a similar man-in-the-middle adversary who reconstructs input images from prediction vectors observed from f .
Revised: [29] assumes a similar man-in-the-middle adversary who reconstructs input images from prediction vectors observed from f .

Original: Our goal is to protect the model explanations from this strong black-box adversary conducting MI attacks.
Revised: Our goal is to protect the model explanations from this strong black-box adversary conducting MI attacks.

Original: 3.2 Model Inversion Attacks Using Model Explanations
Revised: 3.2 Model Inversion Attacks Using Model Explanations

Original: Zhao et al.
Revised: Zhao et al.

Original: [32] proposed a novel black-box MI attack (ExpMI) that exploits prediction vectors generated from a target classifier f as well as the model explanations for each decision.
Revised: [32] proposed a novel black-box MI attack (ExpMI) that exploits prediction vectors generated from a target classifier f as well as the model explanations for each decision.

Original: To reconstruct the input images, they trained a transposed CNN model f A based on auxiliary data, each of which was a prediction vector paired with its model explanation.
Revised: To reconstruct the input images, they trained a transposed CNN model f A based on auxiliary data, each of which was a prediction vector paired with its model explanation.

Original: Consider a target classifier f
Revised: Consider a target classifier f

Original: that is a function f ( x ) → ( e, ˆ y ) that emits a prediction vector and model explanation (i.e., an attribution map).
Revised: that is a function f ( x ) → ( e, ˆ y ) that emits a prediction vector and model explanation (i.e., an attribution map).

Original: The adversary’s goal is to compute f A ( e, ˆ y )
Revised: The adversary’s goal is to compute f A ( e, ˆ y )

Original: → ˆ x , which takes explanation e and ˆ y to produce ˆ x , revealing sensitive attributes of x .
Revised: → ˆ x , which takes explanation e and ˆ y to produce ˆ x , revealing sensitive attributes of x .

Original: Figure 3 illustrates a simplified structure of the ExpMI model.
Revised: Figure 3 illustrates a simplified structure of the ExpMI model.

Original: To take advantage of structural information in activation maps, ExpMI utilizes a U-Net-style bypass connection along with flattened inputs directly appended to the intermediate embedding.
Revised: To take advantage of structural information in activation maps, ExpMI utilizes a U-Net-style bypass connection along with flattened inputs directly appended to the intermediate embedding.

Original: [Table]
Revised: [Table]

Original: _base64="Q5jnLNqgSmQAnb22w0hnW60w3wA=">AAAB7nicbZC7SgNBFIbPxluMt6ilzeIiWIVdi2gZsNEugpsEkhBmJ7PJkNnZZeassCxpfAMbC0VsLX0JX8DOx/ANnFwKTfxh4OP/z2HOOUEiuEbX/bIKK6tr6xvFzdLW9s7uXnn/oKHjVFHm01jEqhUQzQSXzEeOgrUSxUgUCNYMRpeTvHnHlOaxvMUsYd2IDCQPOSVorGZnSDDPxr2y41bcqexl8Obg1Jzvj3v/vVrvlT87/ZimEZNIBdG67bkJdnOikFPBxqVOqllC6IgMWNugJBHT3Xw67tg+MU7fDmNlnkR76v7uyEmkdRYFpjIiONSL2cT8L2unGF50cy6TFJmks4/CVNgY25Pd7T5XjKLIDBCquJnVpkOiCEVzoZI5gre48jI0zipeteLdeE7tGmYqwhEcwyl4cA41uII6+EBhBA/wBM9WYj1aL9brrLRgzXsO4Y+stx/eeJOq</latexit>
Revised: _base64="Q5jnLNqgSmQAnb22w0hnW60w3wA=">AAAB7nicbZC7SgNBFIbPxluMt6ilzeIiWIVdi2gZsNEugpsEkhBmJ7PJkNnZZeassCxpfAMbC0VsLX0JX8DOx/ANnFwKTfxh4OP/z2HOOUEiuEbX/bIKK6tr6xvFzdLW9s7uXnn/oKHjVFHm01jEqhUQzQSXzEeOgrUSxUgUCNYMRpeTvHnHlOaxvMUsYd2IDCQPOSVorGZnSDDPxr2y41bcqexl8Obg1Jzvj3v/vVrvlT87/ZimEZNIBdG67bkJdnOikFPBxqVOqllC6IgMWNugJBHT3Xw67tg+MU7fDmNlnkR76v7uyEmkdRYFpjIiONSL2cT8L2unGF50cy6TFJmks4/CVNgY25Pd7T5XjKLIDBCquJnVpkOiCEVzoZI5gre48jI0zipeteLdeE7tGmYqwhEcwyl4cA41uII6+EBhBA/wBM9WYj1aL9brrLRgzXsO4Y+stx/eeJOq</latexit>

Original: [Figure]
Revised: [Figure]

Original: The adversary takes advantage of e , which leaks sensitive information pertaining to x and drastically improves the reconstruction performance of f A .
Revised: The adversary takes advantage of e , which leaks sensitive information pertaining to x and drastically improves the reconstruction performance of f A .

Original: As Figure 1 shows, the quality of the reconstructed image ˆ x with e is significantly better than the MI attack using prediction vectors alone generates.
Revised: As Figure 1 shows, the quality of the reconstructed image ˆ x with e is significantly better than the MI attack using prediction vectors alone generates.

Original: Considering that model explanation has become instrumental in identifying the reasons behind the decisions of DNN models in various industry sectors, it becomes inevitable for corporations to provide model explanations.
Revised: Considering that model explanation has become instrumental in identifying the reasons behind the decisions of DNN models in various industry sectors, it becomes inevitable for corporations to provide model explanations.

Original: However, providing such information also jeopardizes privacy by revealing sensitive attributes of reconstructed images.
Revised: However, providing such information also jeopardizes privacy by revealing sensitive attributes of reconstructed images.

Original: To this end, we propose GNIME, a novel defense framework that learns to inject noise into model explanations, thus minimizing the privacy risk of ExpMI attacks and maximizing the utility of noise-injected model explanations.
Revised: To this end, we propose GNIME, a novel defense framework that learns to inject noise into model explanations, thus minimizing the privacy risk of ExpMI attacks and maximizing the utility of noise-injected model explanations.

Original:  Learning to Generate Noise for Model Explanations
Revised:  Learning to Generate Noise for Model Explanations

Original: Figure 2 depicts the overall workflow of GNIME.
Revised: Figure 2 depicts the overall workflow of GNIME.

Original: Phase I trains the noise injector and MI attack networks, thus inducing the noise injector network to inject inversion-resistant perturbations into model explanations.
Revised: Phase I trains the noise injector and MI attack networks, thus inducing the noise injector network to inject inversion-resistant perturbations into model explanations.

Original: Phase II then deploys the noise injector network in injecting noise into each model explanation upon its release.
Revised: Phase II then deploys the noise injector network in injecting noise into each model explanation upon its release.

Original: Given a target classifier f ( x ) → ( e, ˆ y ) , GNIME learns to inject noise into e to minimize the MI threat.
Revised: Given a target classifier f ( x ) → ( e, ˆ y ) , GNIME learns to inject noise into e to minimize the MI threat.

Original: Our approach utilizes the noise generator network ( f NG ) and inversion network ( f INV ), both of which are trained upon D T
Revised: Our approach utilizes the noise generator network ( f NG ) and inversion network ( f INV ), both of which are trained upon D T

Original: = { x i , y i } Ni =1 , the original training data of f .
Revised: = { x i , y i } Ni =1 , the original training data of f .

Original: Let f NG ( e, ˆ y )
Revised: Let f NG ( e, ˆ y )

Original: → ˆ e be the NG network that generates a new explanation attribution map ˆ e , which includes noise.
Revised: → ˆ e be the NG network that generates a new explanation attribution map ˆ e , which includes noise.

Original: For the INV network, we design f INV ( e, ˆ y )
Revised: For the INV network, we design f INV ( e, ˆ y )

Original: → ˆ x that takes e and ˆ y , then computes an inversion image ˆ x .
Revised: → ˆ x that takes e and ˆ y , then computes an inversion image ˆ x .

Original: For the loss functions of f NG and f INV , we use an explanation distortion loss ( L xd ) to encourage minimum perturbation to the explanations and an image reconstruction loss ( L re ) to encourage inversion performance.
Revised: For the loss functions of f NG and f INV , we use an explanation distortion loss ( L xd ) to encourage minimum perturbation to the explanations and an image reconstruction loss ( L re ) to encourage inversion performance.

Original: We use the pixel-wise mean squared error (MSE) between e and ˆ e for L xd , and the MSE between x and ˆ x for L re .
Revised: We use the pixel-wise mean squared error (MSE) between e and ˆ e for L xd , and the MSE between x and ˆ x for L re .

Original: Inspired by Generative Adversarial Networks [11], we form a two-player minimax game in which f NG and f INV are trained together using two different objectives.
Revised: Inspired by Generative Adversarial Networks [11], we form a two-player minimax game in which f NG and f INV are trained together using two different objectives.

Original: Specifically, the training phase consists of two steps for each epoch.
Revised: Specifically, the training phase consists of two steps for each epoch.

Original: In the first step, we freeze f NG and train f INV by minimizing the following loss:
Revised: In the first step, we freeze f NG and train f INV by minimizing the following loss:

Original: [Equation]
Revised: [Equation]

Original: The objective is to improve the inversion capability of f INV given ( e, ˆ y ) and (ˆ e, ˆ y ) .
Revised: The objective is to improve the inversion capability of f INV given ( e, ˆ y ) and (ˆ e, ˆ y ) .

Original: Note that L INV includes the reconstruction loss from the clean explanations in order to prevent the network from overfitting to the perturbed explanations.
Revised: Note that L INV includes the reconstruction loss from the clean explanations in order to prevent the network from overfitting to the perturbed explanations.

Original: For the second step, we freeze f INV and train f NG by minimizing the following loss:
Revised: For the second step, we freeze f INV and train f NG by minimizing the following loss:

Original: [Equation]
Revised: [Equation]

Original: Input
Revised: Input

Original: Generate do
Revised: Generate do

Original: Sample a
Revised: Sample a

Original: E Let ˆ e i = f
Revised: E Let ˆ e i = f

Original: ˆ y
Revised: ˆ y

Original: It aims to minimize the distortion between e and ˆ e , while maximizing the error between x and ˆ x .
Revised: It aims to minimize the distortion between e and ˆ e , while maximizing the error between x and ˆ x .

Original: Therefore, we train f NG to generate a noise-injected attribution map that is similar to the original attribution map but causes f INV to generate an inversion image dissimilar from its original image.
Revised: Therefore, we train f NG to generate a noise-injected attribution map that is similar to the original attribution map but causes f INV to generate an inversion image dissimilar from its original image.

Original: Algorithm 1 explains our training process.
Revised: Algorithm 1 explains our training process.

Original: In L NG , we set λ = 500 for CelebA models and λ =for MNIST, CIFAR-10, and ImageNet-100 models, then deploy the final model after 500 epochs.
Revised: In L NG , we set λ = 500 for CelebA models and λ =for MNIST, CIFAR-10, and ImageNet-100 models, then deploy the final model after 500 epochs.

Original: In the worst case, f NG will merely cause a regularization effect on f INV , rendering it robust to any noisy explanation.
Revised: In the worst case, f NG will merely cause a regularization effect on f INV , rendering it robust to any noisy explanation.

Original: However, as f NG is trained over epochs, it learns to capture and suppress features inside explanations that contain exploitable information regarding the original input image.
Revised: However, as f NG is trained over epochs, it learns to capture and suppress features inside explanations that contain exploitable information regarding the original input image.

Original: Once the training is finished, we discard f INV and only make use of f NG for Phase II.
Revised: Once the training is finished, we discard f INV and only make use of f NG for Phase II.

Original: For every attribution map e , we use f NG to generate a noise-injected attribution map ˆ e , then release ˆ e instead of e upon a given query x .
Revised: For every attribution map e , we use f NG to generate a noise-injected attribution map ˆ e , then release ˆ e instead of e upon a given query x .

Original: Additionally, GNIME service providers can inject less noise into the explanations with an arbitrary
Revised: Additionally, GNIME service providers can inject less noise into the explanations with an arbitrary

Original: MSE bound, γ .
Revised: MSE bound, γ .

Original: In this case, GNIME computes a new attribution map e ′
Revised: In this case, GNIME computes a new attribution map e ′

Original: (see Equation 3); we normalize the noise difference n between ˆ e and e by dividing it by the square root of the variance of n over γ and then add this normalized difference to e :
Revised: (see Equation 3); we normalize the noise difference n between ˆ e and e by dividing it by the square root of the variance of n over γ and then add this normalized difference to e :

Original: [Equation]
Revised: [Equation]

Original: n
Revised: n

Original: [Equation]
Revised: [Equation]

Original: σ 2 ( n ) denotes the pixel variance of n .
Revised: σ 2 ( n ) denotes the pixel variance of n .

Original: Therefore, by controlling γ , GNIME service providers can utilize the trade-off and choose to preserve better explanation functionality in exchange for defensive capability.
Revised: Therefore, by controlling γ , GNIME service providers can utilize the trade-off and choose to preserve better explanation functionality in exchange for defensive capability.

Original: Refer to the supplementary material for further analysis regarding the trade-off.
Revised: Refer to the supplementary material for further analysis regarding the trade-off.

Original:  Experiments 5.1 Experimental Setup
Revised:  Experiments 5.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: We evaluate GNIME using four datasets: (1) CelebA [17], (2) MNIST [14], (3) CIFAR10 [13], and (4) ImageNet [6] , each of which is freely available for research purposes.
Revised: We evaluate GNIME using four datasets: (1) CelebA [17], (2) MNIST [14], (3) CIFAR10 [13], and (4) ImageNet [6] , each of which is freely available for research purposes.

Original: For CelebA, we use its balanced subset consisting of 30,000 face images of 1000 individuals.
Revised: For CelebA, we use its balanced subset consisting of 30,000 face images of 1000 individuals.

Original: To obtain a tighter bound for the faces, we crop the center and resize each CelebA image to 128 × 128.
Revised: To obtain a tighter bound for the faces, we crop the center and resize each CelebA image to 128 × 128.

Original: MNIST consists of 70,000 images of handwritten digits, and CIFAR-10 has 60,000 images of 10 different objects.
Revised: MNIST consists of 70,000 images of handwritten digits, and CIFAR-10 has 60,000 images of 10 different objects.

Original: Both
Revised: Both

Original: MNIST and CIFAR-10 images were resized to 32 × 32.
Revised: MNIST and CIFAR-10 images were resized to 32 × 32.

Original: We note that previous studies [29, 31, 32] adopted the same datasets to demonstrate how effectively their frameworks mitigate the MI threat.
Revised: We note that previous studies [29, 31, 32] adopted the same datasets to demonstrate how effectively their frameworks mitigate the MI threat.

Original: In addition, we evaluate GNIME on ImageNet-100 [6], which is a large scale dataset of higher resolution, consisting of 100 classes randomly selected classes from ImageNet; each image is resized to 256 × 256.
Revised: In addition, we evaluate GNIME on ImageNet-100 [6], which is a large scale dataset of higher resolution, consisting of 100 classes randomly selected classes from ImageNet; each image is resized to 256 × 256.

Original: We split each dataset into two disjoint sets of the same size: the target and attack datasets.
Revised: We split each dataset into two disjoint sets of the same size: the target and attack datasets.

Original: The target dataset is used to train a target model f and GNIME.
Revised: The target dataset is used to train a target model f and GNIME.

Original: The attack dataset is used to train the inversion networks for explanation-aware MI attacks and to test GNIME.
Revised: The attack dataset is used to train the inversion networks for explanation-aware MI attacks and to test GNIME.

Original: Specifically, we split this attack dataset with an 80/20 ratio for the train/test split.
Revised: Specifically, we split this attack dataset with an 80/20 ratio for the train/test split.

Original: Note that we assume an adversary capable of training f
Revised: Note that we assume an adversary capable of training f

Original: A upon a dataset of which the underlying distribution is nearly identical to that of the target network’s training data.
Revised: A upon a dataset of which the underlying distribution is nearly identical to that of the target network’s training data.

Original: We intentionally assume this strong adversary to demonstrate how good GNIME is in mitigating the MI threat.
Revised: We intentionally assume this strong adversary to demonstrate how good GNIME is in mitigating the MI threat.

Original: Moreover, we also assess the performance of GNIME against a weaker but more practical adversary using an auxiliary dataset that has a different underlying distribution (i.e., a separately collected set for conducting a similar task) [29].
Revised: Moreover, we also assess the performance of GNIME against a weaker but more practical adversary using an auxiliary dataset that has a different underlying distribution (i.e., a separately collected set for conducting a similar task) [29].

Original: For instance, when attacking the CelebA model, we randomly sample face images from FaceScrub [20] to train f A .
Revised: For instance, when attacking the CelebA model, we randomly sample face images from FaceScrub [20] to train f A .

Original: Note that the overlapping identities from FaceScrub which also appear in CelebA are removed before the sampling process.
Revised: Note that the overlapping identities from FaceScrub which also appear in CelebA are removed before the sampling process.

Original: We extract a tighter bound of each FaceScrub face according to its official bounding box information.
Revised: We extract a tighter bound of each FaceScrub face according to its official bounding box information.

Original: Target Models.
Revised: Target Models.

Original: We implement different target models for different datasets.
Revised: We implement different target models for different datasets.

Original: For CelebA, the target model consists of three convolutional layers followed by two fully-connected (FC) layers.
Revised: For CelebA, the target model consists of three convolutional layers followed by two fully-connected (FC) layers.

Original: We employ dropout before each FC layer to avoid overfitting.
Revised: We employ dropout before each FC layer to avoid overfitting.

Original: For MNIST and CIFAR-10, we use a target model with two convolutional layers followed by two FC layers.
Revised: For MNIST and CIFAR-10, we use a target model with two convolutional layers followed by two FC layers.

Original: We selected the same models that
Revised: We selected the same models that

Original: Zhao et al.
Revised: Zhao et al.

Original: [32] used to demonstrate their MI attacks.
Revised: [32] used to demonstrate their MI attacks.

Original: For ImageNet-100, we used the model from
Revised: For ImageNet-100, we used the model from

Original: CelebA with an additional convolutional layer.
Revised: CelebA with an additional convolutional layer.

Original: Inversion Methods.
Revised: Inversion Methods.

Original: We use two MI attack methods (ExpMI and PredMI) as baselines in our experiments.
Revised: We use two MI attack methods (ExpMI and PredMI) as baselines in our experiments.

Original: Specifically, we compare the efficacy of various defense mechanisms against the stateof-the-art ExpMI attack [32].
Revised: Specifically, we compare the efficacy of various defense mechanisms against the stateof-the-art ExpMI attack [32].

Original: In addition, we consider the prediction-only MI attack using no model explanations (PredMI), suggested by Yang et al.
Revised: In addition, we consider the prediction-only MI attack using no model explanations (PredMI), suggested by Yang et al.

Original: Note that the inversion results via PredMI form an upper bound that represents the ideal case for any defenses in which model explanations leak no additional knowledge that the adversary is able to exploit.
Revised: Note that the inversion results via PredMI form an upper bound that represents the ideal case for any defenses in which model explanations leak no additional knowledge that the adversary is able to exploit.

Original: We conducted each MI attack five times and reported average metrics with standard deviations.
Revised: We conducted each MI attack five times and reported average metrics with standard deviations.

Original: All experiments took place on a system equipped with 512GBs of RAM, two Intel Xeon Gold 6258R CPUs, and four RTX 3090 GPUs.
Revised: All experiments took place on a system equipped with 512GBs of RAM, two Intel Xeon Gold 6258R CPUs, and four RTX 3090 GPUs.

Original: 5.2 Evaluation Metrics
Revised: 5.2 Evaluation Metrics

Original: We assess whether the reconstructed image is close to its original image by leveraging four metrics:
Revised: We assess whether the reconstructed image is close to its original image by leveraging four metrics:

Original: MSE, SSIM, TCA, and DeePSiM, which have been extensively used for quantitative evaluations in previous studies [29, 31, 32].
Revised: MSE, SSIM, TCA, and DeePSiM, which have been extensively used for quantitative evaluations in previous studies [29, 31, 32].

Original: Mean Squared Error (MSE).
Revised: Mean Squared Error (MSE).

Original: The MSE is a metric that measures the distance between a reconstructed image and its corresponding original image.
Revised: The MSE is a metric that measures the distance between a reconstructed image and its corresponding original image.

Original: A small MSE indicates that the reconstructed image is close to its original image.
Revised: A small MSE indicates that the reconstructed image is close to its original image.

Original: For normalization, input pixel values of the inputs are scaled to be within the range of [0 , 1] .
Revised: For normalization, input pixel values of the inputs are scaled to be within the range of [0 , 1] .

Original: Structural Similarity Index Measure (SSIM).
Revised: Structural Similarity Index Measure (SSIM).

Original: The SSIM computes the distortion of structural information between two images [28].
Revised: The SSIM computes the distortion of structural information between two images [28].

Original: Since humans are good at recognizing structural differences between two images, this metric shows human perceptual similarity between two images.
Revised: Since humans are good at recognizing structural differences between two images, this metric shows human perceptual similarity between two images.

Original: An SSIM score spans from 0 to 1, and the higher score indicates better reconstruction.
Revised: An SSIM score spans from 0 to 1, and the higher score indicates better reconstruction.

Original: Deep Perceptual Similarity Metric (DeePSiM).
Revised: Deep Perceptual Similarity Metric (DeePSiM).

Original: The DeePSiM shows the similarity between two images at an intermediate representation level [7].
Revised: The DeePSiM shows the similarity between two images at an intermediate representation level [7].

Original: Unlike SSIM, which illustrates perceptual similarity, DeePSiM explains perceptual similarity from the neural network perspective.
Revised: Unlike SSIM, which illustrates perceptual similarity, DeePSiM explains perceptual similarity from the neural network perspective.

Original: Specifically, we trained a separate evaluation classifier f E , which shares the same architecture and training dataset with the target model.
Revised: Specifically, we trained a separate evaluation classifier f E , which shares the same architecture and training dataset with the target model.

Original: We then computed the feature distance between intermediate results from the penultimate layer of f E , comparing the high-level features identified by f E [32].
Revised: We then computed the feature distance between intermediate results from the penultimate layer of f E , comparing the high-level features identified by f E [32].

Original: If f INV accurately reconstructs the high-level image features, it results in a high DeePSiM score.
Revised: If f INV accurately reconstructs the high-level image features, it results in a high DeePSiM score.

Original: Transferable Classification Accuracy (TCA).
Revised: Transferable Classification Accuracy (TCA).

Original: We also measure a TCA to evaluate whether inverted images are generally recognizable by another DNN classifier of which the structure is the same as that of the target model.
Revised: We also measure a TCA to evaluate whether inverted images are generally recognizable by another DNN classifier of which the structure is the same as that of the target model.

Original: This metric indirectly shows whether the reconstructed image incorporates the key features of its original image.
Revised: This metric indirectly shows whether the reconstructed image incorporates the key features of its original image.

Original: We measured this metric using f E .
Revised: We measured this metric using f E .

Original: Note that f E would correctly predict the reconstructed image if the reconstructed image successfully incorporates the key features of the original image, rather than to overfit to unimportant details captured only by the target classifier.
Revised: Note that f E would correctly predict the reconstructed image if the reconstructed image successfully incorporates the key features of the original image, rather than to overfit to unimportant details captured only by the target classifier.

Original: 5.3 Experiment Results 5.3.1 Defending against MI Attacks
Revised: 5.3 Experiment Results 5.3.1 Defending against MI Attacks

Original: We evaluate the performance of vanilla explanation-aware MI attacks and then measure the efficacy of GNIME in decreasing information leakage from such MI attacks.
Revised: We evaluate the performance of vanilla explanation-aware MI attacks and then measure the efficacy of GNIME in decreasing information leakage from such MI attacks.

Original: Specifically, we compare five settings (two baseline attacks and three defenses against ExpMI attacks): (1) MI attacks using the model’s prediction outputs alone (PredMI); (2) MI attacks using both prediction outputs and model explanations (ExpMI); (3) ExpMI attacks against a defense that injects Gaussian noise into saliency maps (RND); (4) ExpMI attacks against a defense that injects optimized noise into saliency maps
Revised: Specifically, we compare five settings (two baseline attacks and three defenses against ExpMI attacks): (1) MI attacks using the model’s prediction outputs alone (PredMI); (2) MI attacks using both prediction outputs and model explanations (ExpMI); (3) ExpMI attacks against a defense that injects Gaussian noise into saliency maps (RND); (4) ExpMI attacks against a defense that injects optimized noise into saliency maps

Original: [Table]
Revised: [Table]

Original: C
Revised: C

Original: I
Revised: I

Original: F AR -
Revised: F AR -

Original: [Table]
Revised: [Table]

Original: (OND); and (5) ExpMI attacks against GNIME.
Revised: (OND); and (5) ExpMI attacks against GNIME.

Original: When measuring the defensive capability of RND,
Revised: When measuring the defensive capability of RND,

Original: OND, and GNIME, each MI network was separately trained using the explanations perturbed by the corresponding defense method.
Revised: OND, and GNIME, each MI network was separately trained using the explanations perturbed by the corresponding defense method.

Original: This resembles the real-world adversary in our black-box setting, who has no way of obtaining clean model explanations.
Revised: This resembles the real-world adversary in our black-box setting, who has no way of obtaining clean model explanations.

Original: Table 1 compares the performance of GNIME against these experimental settings assuming a target
Revised: Table 1 compares the performance of GNIME against these experimental settings assuming a target

Original: ML service providing Grad, Grad-CAM, and LRP, respectively.
Revised: ML service providing Grad, Grad-CAM, and LRP, respectively.

Original: Recall from §3 that we assume a strong adversary in possession of auxiliary data of which the distribution is the same as the training dataset of the target model.
Revised: Recall from §3 that we assume a strong adversary in possession of auxiliary data of which the distribution is the same as the training dataset of the target model.

Original: We thus use the train split of our attack dataset for training GNIME and the test split for measuring the metrics (§5.1).
Revised: We thus use the train split of our attack dataset for training GNIME and the test split for measuring the metrics (§5.1).

Original: Considering that ExpMI shows better reconstruction quality compared to PredMI in all the cases, exploiting model explanations to perform MI attacks indeed contributes to additional information leakage.
Revised: Considering that ExpMI shows better reconstruction quality compared to PredMI in all the cases, exploiting model explanations to perform MI attacks indeed contributes to additional information leakage.

Original: Surprisingly, the SSIM score significantly boosted from 0.1688 to 0.8862 when targeting CIFAR-10-LRP.
Revised: Surprisingly, the SSIM score significantly boosted from 0.1688 to 0.8862 when targeting CIFAR-10-LRP.

Original: These results demonstrate the necessity of defense against ExpMI attacks.
Revised: These results demonstrate the necessity of defense against ExpMI attacks.

Original: We now compare the performance of GNIME against other two defense settings (RND and OND) that add noise to achieve the same defensive goal.
Revised: We now compare the performance of GNIME against other two defense settings (RND and OND) that add noise to achieve the same defensive goal.

Original: RND corresponds to a naive approach that randomly adds Gaussian noise, which serves as our defense baseline.
Revised: RND corresponds to a naive approach that randomly adds Gaussian noise, which serves as our defense baseline.

Original: For OND, instead of training f NG , we use the projected gradient descent [18] to optimize the input image towards increasing the loss function of f INV .
Revised: For OND, instead of training f NG , we use the projected gradient descent [18] to optimize the input image towards increasing the loss function of f INV .

Original: Note that adding huge noise can prevent information leakage, but it would undermine the ability to explain the model’s decisions.
Revised: Note that adding huge noise can prevent information leakage, but it would undermine the ability to explain the model’s decisions.

Original: Hence, for fair comparison, we clip the perturbation magnitude of RND and OND to align with the average perturbation size of GNIME.
Revised: Hence, for fair comparison, we clip the perturbation magnitude of RND and OND to align with the average perturbation size of GNIME.

Original: Target Image w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: (a) ExpMI against Grad
Revised: (a) ExpMI against Grad

Original: Target Image w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: (b) ExpMI against Grad-CAM
Revised: (b) ExpMI against Grad-CAM

Original: Target Image w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: (c) ExpMI against LRP
Revised: (c) ExpMI against LRP

Original: As shown in the table, GNIME succeeds in decreasing the information leakage threat for all the explanations and data types.
Revised: As shown in the table, GNIME succeeds in decreasing the information leakage threat for all the explanations and data types.

Original: Although RND also slightly alleviates the MI threat by degrading the inversion quality, GNIME significantly outperforms RND in all the cases.
Revised: Although RND also slightly alleviates the MI threat by degrading the inversion quality, GNIME significantly outperforms RND in all the cases.

Original: Furthermore, GNIME shows superior performance compared to OND as well in all cases.
Revised: Furthermore, GNIME shows superior performance compared to OND as well in all cases.

Original: For example, when targeting
Revised: For example, when targeting

Original: CelebA-LRP, RND and OND lower the TCA by 0.0588 and 0.0940, respectively.
Revised: CelebA-LRP, RND and OND lower the TCA by 0.0588 and 0.0940, respectively.

Original: On the other hand,
Revised: On the other hand,

Original: GNIME decreases the TCA by 0.1821, which corresponds to 3.10 × of RND and 1.94 × of OND.
Revised: GNIME decreases the TCA by 0.1821, which corresponds to 3.10 × of RND and 1.94 × of OND.

Original: From these observations, we conclude that noise optimized against the f INV fails to generalize on f A , which is independently trained for ExpMI attacks, and it requires a dedicated model f NG to generate effective inversion-resistant noise.
Revised: From these observations, we conclude that noise optimized against the f INV fails to generalize on f A , which is independently trained for ExpMI attacks, and it requires a dedicated model f NG to generate effective inversion-resistant noise.

Original: PredMI serves as an upper bound for all the presented defense frameworks, which shows the case when there exists no additional information leakage threat that stems from providing model explanations (§5.1).
Revised: PredMI serves as an upper bound for all the presented defense frameworks, which shows the case when there exists no additional information leakage threat that stems from providing model explanations (§5.1).

Original: Considering this upper bound shown in the table, we believe that the degree of GNIME achieving its defensive goal is close to its upper bound.
Revised: Considering this upper bound shown in the table, we believe that the degree of GNIME achieving its defensive goal is close to its upper bound.

Original: For instance, the DeePSiM score against
Revised: For instance, the DeePSiM score against

Original: PredMI and ExpMI targeting CelebA-Grad with GNIME achieves 0.1851 and 0.1880, respectively.
Revised: PredMI and ExpMI targeting CelebA-Grad with GNIME achieves 0.1851 and 0.1880, respectively.

Original: Figure 4 presents the qualitative experimental results.
Revised: Figure 4 presents the qualitative experimental results.

Original: We observe that GNIME outperforms both
Revised: We observe that GNIME outperforms both

Original: RND and OND in decreasing the reconstruction quality.
Revised: RND and OND in decreasing the reconstruction quality.

Original: When comparing all columns between the third and fifth for each explanation type, GNIME clearly contributes to degrading the quality of reconstructed images; ExpMI still reconstructs most features in the inversion images when the explanations are under the protection of RND or OND, contrary to the inversion images with GNIME.
Revised: When comparing all columns between the third and fifth for each explanation type, GNIME clearly contributes to degrading the quality of reconstructed images; ExpMI still reconstructs most features in the inversion images when the explanations are under the protection of RND or OND, contrary to the inversion images with GNIME.

Original: We further compare the inversion results with a more practical adversary.
Revised: We further compare the inversion results with a more practical adversary.

Original: Here, the adversary cannot train its inversion network f A with the same dataset as the target model (CelebA).
Revised: Here, the adversary cannot train its inversion network f A with the same dataset as the target model (CelebA).

Original: Instead, it draws the training data from another generic distribution.
Revised: Instead, it draws the training data from another generic distribution.

Original: Specifically, we use a subset of FaceScrub to train f A .
Revised: Specifically, we use a subset of FaceScrub to train f A .

Original: This auxiliary training set is arguably easier to obtain than the original dataset [29].
Revised: This auxiliary training set is arguably easier to obtain than the original dataset [29].

Original: Table 2 summarizes the ExpMI attack performance with and without GNIME.
Revised: Table 2 summarizes the ExpMI attack performance with and without GNIME.

Original: The metrics consistently report lower reconstruction performance even when the adversary leverages FaceScrub as the auxiliary dataset.
Revised: The metrics consistently report lower reconstruction performance even when the adversary leverages FaceScrub as the auxiliary dataset.

Original: Interestingly, the defensive capability of GNIME was even greater in the practical setting (e.g., using FaceScrub to attack CelebA-LRP, the degradation of TCA due to GNIME increased from 84.82% degradation (0.2147 → 0.0326) to 99.43% (0.2086 → 0.0012)).
Revised: Interestingly, the defensive capability of GNIME was even greater in the practical setting (e.g., using FaceScrub to attack CelebA-LRP, the degradation of TCA due to GNIME increased from 84.82% degradation (0.2147 → 0.0326) to 99.43% (0.2086 → 0.0012)).

Original: These results demonstrate that
Revised: These results demonstrate that

Original: GNIME is more effective against the practical adversary of conducting ExpMI attacks.
Revised: GNIME is more effective against the practical adversary of conducting ExpMI attacks.

Original: 5.3.2 Preserving XAI Functionality
Revised: 5.3.2 Preserving XAI Functionality

Original: In this section, we further evaluate whether an attribution map ˆ e noised by GNIME still provides interpretable explanations on the model’s decisions, thus preserving its functionality.
Revised: In this section, we further evaluate whether an attribution map ˆ e noised by GNIME still provides interpretable explanations on the model’s decisions, thus preserving its functionality.

Original: The XAI methods (e.g., Grad, Grad-CAM, and LRP) produce an attribution map that reflects the degree of importance for each pixel contributing to a target model emitting a decision.
Revised: The XAI methods (e.g., Grad, Grad-CAM, and LRP) produce an attribution map that reflects the degree of importance for each pixel contributing to a target model emitting a decision.

Original: Therefore, an input image multiplied by this model explanation ( x ⊙ e ) erodes unimportant pixels and leaves important pixels intact.
Revised: Therefore, an input image multiplied by this model explanation ( x ⊙ e ) erodes unimportant pixels and leaves important pixels intact.

Original: That is, we can indirectly evaluate whether the original functionality of a model explanation e is preserved in its perturbed version ˆ e by comparing the pixels left in x ⊙ e and x ⊙ ˆ e using DeePSiM.
Revised: That is, we can indirectly evaluate whether the original functionality of a model explanation e is preserved in its perturbed version ˆ e by comparing the pixels left in x ⊙ e and x ⊙ ˆ e using DeePSiM.

Original: We compare the intermediate representation values in the penultimate layer of the target classifier
Revised: We compare the intermediate representation values in the penultimate layer of the target classifier

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: f for each input: x ⊙ e and x ⊙ ˆ e .
Revised: f for each input: x ⊙ e and x ⊙ ˆ e .

Original: A higher DeePSiM score indicates a higher resemblance in the high-level features captured by the target model.
Revised: A higher DeePSiM score indicates a higher resemblance in the high-level features captured by the target model.

Original: Table 3 shows the average DeePSiM value of all x ⊙ e and x ⊙ ˆ e pairs in which each x is from the test split of the attack dataset.
Revised: Table 3 shows the average DeePSiM value of all x ⊙ e and x ⊙ ˆ e pairs in which each x is from the test split of the attack dataset.

Original: The model perceptual similarity between x ⊙ e and x ⊙ ˆ e is at least 0.6734, 0.9298, 0.8761, and 0.9097 for the CelebA, MNIST, CIFAR-10, and ImageNet-100 models, respectively.
Revised: The model perceptual similarity between x ⊙ e and x ⊙ ˆ e is at least 0.6734, 0.9298, 0.8761, and 0.9097 for the CelebA, MNIST, CIFAR-10, and ImageNet-100 models, respectively.

Original: This indicates that the original functionality of e largely remains intact.
Revised: This indicates that the original functionality of e largely remains intact.

Original: [Figure]
Revised: [Figure]

Original: Perturbation Magnitude.
Revised: Perturbation Magnitude.

Original: We analyze the distribution of MSEs between e and ˆ e pairs for
Revised: We analyze the distribution of MSEs between e and ˆ e pairs for

Original: CelebA depicted in Figure 5.
Revised: CelebA depicted in Figure 5.

Original: The figure shows two triples ( x , e , ˆ e ), each of which respectively represents a common case and an extreme case of perturbation magnitude (i.e., MSE = 0.
Revised: The figure shows two triples ( x , e , ˆ e ), each of which respectively represents a common case and an extreme case of perturbation magnitude (i.e., MSE = 0.

Original: and MSE = 0.0055).
Revised: and MSE = 0.0055).

Original: Note that even the perturbation of the extreme case is hardly perceptible.
Revised: Note that even the perturbation of the extreme case is hardly perceptible.

Original: These results suggest that GNIME applies infinitesimal perturbations to e such that it preserves its original functionality.
Revised: These results suggest that GNIME applies infinitesimal perturbations to e such that it preserves its original functionality.

Original: We include further visualization of perturbed model explanations in the supplementary material.
Revised: We include further visualization of perturbed model explanations in the supplementary material.

Original: In addition, recall from §4 that one can further vary γ in Equation 3 to control the perturbation magnitude.
Revised: In addition, recall from §4 that one can further vary γ in Equation 3 to control the perturbation magnitude.

Original: We observed a linear trade-off relationship between the defensive capability and the explanation functionality of GNIME.
Revised: We observed a linear trade-off relationship between the defensive capability and the explanation functionality of GNIME.

Original: GNIME service providers can use γ of their choice with this in mind.
Revised: GNIME service providers can use γ of their choice with this in mind.

Original: When choosing to use 0.0040 for γ , GNIME achieves the 0.6340 DeePSiM.
Revised: When choosing to use 0.0040 for γ , GNIME achieves the 0.6340 DeePSiM.

Original: However, when limiting the perturbation size to 0.0005, the DeePSiM score increases to 0.8497.
Revised: However, when limiting the perturbation size to 0.0005, the DeePSiM score increases to 0.8497.

Original: Please refer to the supplementary material for further experimental results.
Revised: Please refer to the supplementary material for further experimental results.

Original: 5.3.3 Investigating Additional Information Leakage.
Revised: 5.3.3 Investigating Additional Information Leakage.

Original: Although GNIME significantly lowers the risk of information leakage to the degree to that of PredMI, information leakage still remains to some extent even after its deployment.
Revised: Although GNIME significantly lowers the risk of information leakage to the degree to that of PredMI, information leakage still remains to some extent even after its deployment.

Original: However, we stress that
Revised: However, we stress that

Original: GNIME focuses on preventing additional information leakage that stems from model explanations; therefore, the inversion results of PredMI form an upper bound of our defense.
Revised: GNIME focuses on preventing additional information leakage that stems from model explanations; therefore, the inversion results of PredMI form an upper bound of our defense.

Original: Note that GNIME is capable of obscuring several key attributes in the original images, providing inversion results similar to those of PredMI.
Revised: Note that GNIME is capable of obscuring several key attributes in the original images, providing inversion results similar to those of PredMI.

Original: For instance, Figure 1 shows that GNIME removes critical features in the reconstructed images, such as sunglasses, cap, or gender information.
Revised: For instance, Figure 1 shows that GNIME removes critical features in the reconstructed images, such as sunglasses, cap, or gender information.

Original: We additionally evaluate whether GNIME is able to obscure several key attributes of reconstructed images.
Revised: We additionally evaluate whether GNIME is able to obscure several key attributes of reconstructed images.

Original: Similar to Chen et al.
Revised: Similar to Chen et al.

Original: [5], we trained attribute classifiers using CelebA and evaluated whether the attribute classifiers is able to correctly predict attributes when the reconstructed images are provided.
Revised: [5], we trained attribute classifiers using CelebA and evaluated whether the attribute classifiers is able to correctly predict attributes when the reconstructed images are provided.

Original: Specifically, we trained eight attribute classifiers, employing the ResNet50V2 architecture from He et al.
Revised: Specifically, we trained eight attribute classifiers, employing the ResNet50V2 architecture from He et al.

Original: [12] with its final layer replaced for binary classification.
Revised: [12] with its final layer replaced for binary classification.

Original: Note that the training set used to train these classifiers is disjoint from that we used in the input reconstruction.
Revised: Note that the training set used to train these classifiers is disjoint from that we used in the input reconstruction.

Original: We report the
Revised: We report the

Original: F1 score to measure the performance of these attribute classifiers.
Revised: F1 score to measure the performance of these attribute classifiers.

Original: Figure 6 summarizes the evaluation results.
Revised: Figure 6 summarizes the evaluation results.

Original: GNIME performed best in terms of deteriorating the attribute prediction performance for all
Revised: GNIME performed best in terms of deteriorating the attribute prediction performance for all

Original: [Figure]
Revised: [Figure]

Original: attribute types.
Revised: attribute types.

Original: On average, ExpMI without defense reports an F1 score of 0.7672.
Revised: On average, ExpMI without defense reports an F1 score of 0.7672.

Original: Note that GNIME decreases it to 0.3048, which is close to the F1 score of PredMI.
Revised: Note that GNIME decreases it to 0.3048, which is close to the F1 score of PredMI.

Original:  Discussions and Limitations
Revised:  Discussions and Limitations

Original: Constrained Obfuscation.
Revised: Constrained Obfuscation.

Original: The concept of obfuscating sensitive information while retaining useful information has been explored in various fields in establishing privacy [4, 16] and fairness[30].
Revised: The concept of obfuscating sensitive information while retaining useful information has been explored in various fields in establishing privacy [4, 16] and fairness[30].

Original: Their common approach is to obfuscate given input instances and then jointly train a target model to retain a high performance even for those obfuscated instances.
Revised: Their common approach is to obfuscate given input instances and then jointly train a target model to retain a high performance even for those obfuscated instances.

Original: By constast, GNIME does not directly alter the target model paramaters and focuses only on obfuscating model explanations to minimize the MI threat.
Revised: By constast, GNIME does not directly alter the target model paramaters and focuses only on obfuscating model explanations to minimize the MI threat.

Original: This enables the deployment of legacy models without any retraining steps while preversing the orignal model performance.
Revised: This enables the deployment of legacy models without any retraining steps while preversing the orignal model performance.

Original: Model Extraction.
Revised: Model Extraction.

Original: Zhao et al.
Revised: Zhao et al.

Original: [32] proposed an ExpMI attack even when a target DNN model does not provide any explanations (ExpMI-3), which we do not assume in the threat model.
Revised: [32] proposed an ExpMI attack even when a target DNN model does not provide any explanations (ExpMI-3), which we do not assume in the threat model.

Original: The core idea of ExpMI-3 is to build a high-performing surrogate model through a successful model extraction (ME) attack and then construct surrogate explanations based on this surrogate model.
Revised: The core idea of ExpMI-3 is to build a high-performing surrogate model through a successful model extraction (ME) attack and then construct surrogate explanations based on this surrogate model.

Original: However, we stress that once the adversary successfully constructs the surrogate model, the surrogate model exposes the target model to numerous white-box attacks, not only limited to the ExpMI threat.
Revised: However, we stress that once the adversary successfully constructs the surrogate model, the surrogate model exposes the target model to numerous white-box attacks, not only limited to the ExpMI threat.

Original: We argue that a defense against ExpMI-3 should focus on mitigating the ME threat.
Revised: We argue that a defense against ExpMI-3 should focus on mitigating the ME threat.

Original: We implemented a simple ME defense, reverse sigmoid perturbation (RSP)
Revised: We implemented a simple ME defense, reverse sigmoid perturbation (RSP)

Original: [15], and evaluted its performance in terms of degrading the inversion peformance of ExpMI-3; the SSIM score of ExpMI-3 attack against RSP-equipped
Revised: [15], and evaluted its performance in terms of degrading the inversion peformance of ExpMI-3; the SSIM score of ExpMI-3 attack against RSP-equipped

Original: CelebA was 0.4928, which is comparable to the SSIM of ExpMI attack against GNIME-equipped
Revised: CelebA was 0.4928, which is comparable to the SSIM of ExpMI attack against GNIME-equipped

Original: CelebA-LRP, 0.4960.
Revised: CelebA-LRP, 0.4960.

Original: Refer to the further experimental results in the supplementary material.
Revised: Refer to the further experimental results in the supplementary material.

Original: Black-box Adversary.
Revised: Black-box Adversary.

Original: Previous MI attacks [5, 31] have been proposed assuming a white-box adversary who can access the gradients of a target model.
Revised: Previous MI attacks [5, 31] have been proposed assuming a white-box adversary who can access the gradients of a target model.

Original: By contrast, we assume the black-box adversary conducting ExpMI attacks, following Zhao et al.
Revised: By contrast, we assume the black-box adversary conducting ExpMI attacks, following Zhao et al.

Original: Therefore, GNIME is designed to mitigate the ExpMI threat in a black-box manner in which a target model returns its explanation and output upon a given input.
Revised: Therefore, GNIME is designed to mitigate the ExpMI threat in a black-box manner in which a target model returns its explanation and output upon a given input.

Original: Considering that many Machine-Learning-as-a-Service (MLaaS) services operate in a black-box setting, we emphasize that addressing the ExpMI threat in this setting is still important.
Revised: Considering that many Machine-Learning-as-a-Service (MLaaS) services operate in a black-box setting, we emphasize that addressing the ExpMI threat in this setting is still important.

Original: To our knowledge, GNIME is the first defense approach to effectively mitigate the ExpMI threat.
Revised: To our knowledge, GNIME is the first defense approach to effectively mitigate the ExpMI threat.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose the first defense framework against MI attacks using model explanations.
Revised: In this paper, we propose the first defense framework against MI attacks using model explanations.

Original: In
Revised: In

Original: GNIME, we put together the noise generator and inversion networks so that they compete with each other; the noise generator network learns to inject imperceptible noise into model explanations that undermine model inversion, and the inversion network learns to excel at reconstructing the original inputs from model explanations.
Revised: GNIME, we put together the noise generator and inversion networks so that they compete with each other; the noise generator network learns to inject imperceptible noise into model explanations that undermine model inversion, and the inversion network learns to excel at reconstructing the original inputs from model explanations.

Original: GNIME then leverages the trained noise generator network for every model explanation upon its release.
Revised: GNIME then leverages the trained noise generator network for every model explanation upon its release.

Original: Our evaluations across diverse datasets and explanation types demonstrate that GNIME is effective in mitigating the privacy threat that MI attacks impose while preserving the original functionality of model explanations.
Revised: Our evaluations across diverse datasets and explanation types demonstrate that GNIME is effective in mitigating the privacy threat that MI attacks impose while preserving the original functionality of model explanations.

Original: Acknowledgements.
Revised: Acknowledgements.

Original: This work was supported by  Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-00153, Penetration
Revised: This work was supported by the Institute of Information & communications Technology Planning & evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-00153, Penetration

Original: Security Testing of ML Model Vulnerabilities and Defense).
Revised: Security Testing of ML Model Vulnerabilities and Defense).