Original: Learning to Generate Inversion-Resistant Model Explanations
Revised: Learning to Generate Inversion-Resistant Model Explanations

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: The wide adoption of deep neural networks  in mission-critical applications hasspurred the need for interpretable models that provide explanations of the model’sdecisions.
Revised: The wide adoption of deep neural networks (DNNs) in mission-critical applications has spurred the need for interpretable models that provide explanations of the model’s decisions.

Original: Unfortunately, previous studies have demonstrated that model explanations facilitate information leakage, rendering DNN models vulnerable to modelinversion attacks.
Revised: Unfortunately, previous studies have demonstrated that model explanations facilitate information leakage, rendering DNN models vulnerable to model inversion attacks.

Original: These attacks enable the adversary to reconstruct original imagesbased on model explanations, thus leaking privacy-sensitive features.
Revised: These attacks enable the adversary to reconstruct original images based on model explanations, thus leaking privacy-sensitive features.

Original: To this end,we present Generative Noise Injector for Model Explanations (GNIME), a noveldefense framework that perturbs model explanations to minimize the risk of modelinversion attacks while preserving the interpretabilities of the generated explanations.
Revised: To this end, we present Generative Noise Injector for Model Explanations (GNIME), a novel defense framework that perturbs model explanations to minimize the risk of model inversion attacks while preserving the interpretabilities of the generated explanations.

Original: Specifically, we formulate the defense training as a two-player minimaxgame between the inversion attack network on the one hand, which aims to invertmodel explanations, and the noise generator network on the other, which aims toinject perturbations to tamper with model inversion attacks.
Revised: Specifically, we formulate the defense training as a two-player minimax game between the inversion attack network on the one hand, which aims to invert model explanations, and the noise generator network on the other, which aims to inject perturbations to tamper with model inversion attacks.

Original: We demonstrate that GNIME significantly decreases the information leakage in model explanations,decreasing transferable classification accuracy in facial recognition models by upto 84.8% while preserving the original functionality of model explanations.
Revised: We demonstrate that GNIME significantly decreases the information leakage in model explanations, decreasing transferable classification accuracy in facial recognition models by up to 84.8% while preserving the original functionality of model explanations.

Original:  Introduction
Revised:  Introduction

Original: The recent widespread adoption of deep neural networks (DNNs) in building practical real-worldapplications has led to a surge of interest in explainable AI (XAI) since mission-critical tasks (e.g.,medical diagnosis) often request explanations of the model’s predictions.
Revised: The recent widespread adoption of deep neural networks (DNNs) in building practical real-world applications has led to a surge of interest in explainable AI (XAI) since mission-critical tasks (e g., medical diagnosis) often request explanations of the model’s predictions.

Original: Such explanations arenot only helpful in debugging the model but also in providing explanations to end-users on howpredictions are made.
Revised: Such explanations are not only helpful in debugging the model but also in providing explanations to end-users on how predictions are made.

Original: They are often provided in the form of a feature attribution map, which showsthe importance of each feature computed based on the gradients of a target model.
Revised: They are often provided in the form of a feature attribution map which shows the importance of each feature computed based on the gradients of a target model.

Original: Unfortunately, recent studies have demonstrated that such explanations cause additional informationleakage, which can be exploited by an adversary [1, 6, 15, 18, 27].
Revised: Unfortunately, recent studies have demonstrated that such explanations cause additional information leakage, which can be exploited by an adversary [1, 8, 19, 22, 32].

Original: Specifically, Zhao et al.
Revised: Specifically, Zhao et al.

Original: [27]presented a novel black-box model inversion (MI) attack which reconstructs an input image givenits prediction vector and the model explanation from a target DNN model.
Revised: [32] presented a novel black-box model inversion (MI) attack which reconstructs an input image given its prediction vector and the model explanation from a target DNN model.

Original: They demonstrated thatadditional information leaked through model explanations (e.g., Grad or Grad-CAM) increases theperformance of MI attacks by a factor of 2.4 in terms of transferable classification accuracy, enablinghigh-fidelity attacks.
Revised: They demonstrated that additional information leaked through model explanations (e.g., Grad or Grad-CAM) increases the performance of MI attacks by a factor of 2.4 in terms of transferable classification accuracy, enabling high-fidelity attacks.

Original: Figure 1 compares the result of MI attacks using model predictions alone (PredMI) and explanationaware MI attacks using model predictions and explanations together (ExpMI).
Revised: Figure 1 compares the result of MI attacks using model predictions alone (PredMI) and explanationaware MI attacks using model predictions and explanations together (ExpMI).

Original: The examples reconstructed via ExpMI show high-quality faces compared to those via PredMI, leaking additionalinformation, such as facial expressions, and accessories.
Revised: The examples reconstructed via ExpMI show high-quality faces compared to those via PredMI, leaking additional information, such as facial expressions and accessories.

Original: However, previous works have only proposedthe attacks exploiting the information leakage, without providing any defenses against them.
Revised: However, previous works have only proposed the attacks exploiting the information leakage, without providing any defenses against them.

Original: [Figure]

Original: Thus, to defend against MI attacks exploiting model explanations, we propose GNIME,which generates inversion-resistant model explanations.
Revised: Thus, to defend against MI attacks exploiting model explanations, we propose GNIME, which generates inversion-resistant model explanations.

Original: Figure 2 shows the overall GNIMEpipeline.
Revised: Figure 2 shows the overall GNIME pipeline.

Original: GNIME consists of two phases: training and deployment.
Revised: GNIME consists of two phases: training and deployment.

Original: In the training phase, GNIME trains a noise generator that injects perturbations into the explanations provided bythe model under protection to minimize the MIthreat.
Revised: In the training phase, GNIME trains a noise generator that injects perturbations into the explanations provided by the model under protection to minimize the MI threat.

Original: To train this noise generator, GNIMEtrains an MI attack network to compete againsteach other.
Revised: To train this noise generator, GNIME trains an MI attack network to compete against each other.

Original: Then, in the deployment phase, wedeploy the trained noise generator to generateperturbations for the explanations generated foreach test instance. 
Revised: Then, in the deployment phase, we deploy the trained noise generator to obfuscate the explanations generated for each test instance. [Figure]

Original: The key idea of our approach is to establisha two-player minimax game between the MIattack (INV) network and the noise generator(NG) network.
Revised: The key idea of our approach is to establish a two-player minimax game between the MI attack (INV) network and the noise generator (NG) network.

Original: We design the NG network to inject perturbations into the model’s explanations(i.e. , attribution map); the noise-injected modelexplanation is then fed into the INV network to reconstruct the inverted image.
Revised: We design the NG network to inject perturbations into the model’s explanations (i.e., attribution map); the noise-injected model explanation is then fed into the INV network to reconstruct the inverted image.

Original: GNIME conducts analternating training of these networks.
Revised: GNIME conducts an alternating training of these networks.

Original: During the NG’s turn, the NG network is trained to tamperwith the inversion performance of the INV network while minimizing the perturbations to the modelexplanations.
Revised: During the NG’s turn, the NG network is trained to tamper with the inversion performance of the INV network while minimizing the perturbations to the model explanations.

Original: During the INV’s turn, we train the INV network to maximize its model inversion performance.
Revised: During the INV’s turn, we train the INV network to maximize its model inversion performance.

Original: These alternating training procedures improve the INV network to better invert the givenmodel’s explanations while the NG network learns to inject perceptually imperceptible inversionresistant perturbations to the provided explanations.
Revised: These alternating training procedures improve the INV network to better invert the given model’s explanations while the NG network learns to inject perceptually imperceptible inversion-resistant perturbations to the provided explanations.

Original: After training is complete, the GNIME ownerdeploys the NG network to inject perturbations into the original explanations before releasing them.
Revised: After training is complete, the GNIME owner deploys the NG network to inject perturbations into the original explanations before releasing them.

Original: We evaluate the efficacy of GNIME, by measuring 1) how effective the model is in decreasing therisk of information leakage from the model explanations and 2) how well the perturbed explanationspreserve the semantics of the original explanations.
Revised: We evaluate the efficacy of GNIME, by measuring 1) how effective the model is in decreasing the risk of information leakage from the model explanations and 2) how well the perturbed explanations preserve the semantics of the original explanations.

Original: For different types of explanations (i.e., Grad [20],
Revised: For different types of explanations (i.e., Grad [24],

Original: Grad-CAM [17], and LRP [3]), we demonstrate that GNIME consistently generates infinitesimalinversion-resistant noises that significantly impede explanation-aware MI attacks.
Revised: Grad-CAM [21], and LRP [3]), we demonstrate that GNIME consistently generates infinitesimal inversion-resistant noises that significantly impede explanation-aware MI attacks.

Revised: To facilitate further research, we publish GNIME at https://github.com/WSP-LAB/GNIME .

Original: In summary, our contributions are as follows:
Revised: In summary, our contributions are as follows:

Original: • We propose the very first defense framework against explanation-aware MI attacks, whichlearns to suppress inversion-critical features from model explanations.
Revised: • We propose the very first defense framework against explanation-aware MI attacks, which learns to suppress inversion-critical features from model explanations.

Original: • We evaluate GNIME on diverse datasets and demonstrate its efficacy in decreasing the MIrisk while preserving the interpretability of the original explanations.
Revised: • We evaluate GNIME on diverse datasets and demonstrate its efficacy in decreasing the MI risk while preserving the interpretability of the original explanations.

Original:  Related Work
Revised:  Related Work

Original: Explainable AI for CNN models.
Revised: Explainable AI for CNN models.

Original: Previous XAI research has introduced diverse methods ofcomputing an attribution map (i.e., saliency map) that explains the decision of a DNN classifier.
Revised: Previous XAI research has introduced diverse methods of computing an attribution map (i.e., saliency map) that explains the decision of a DNN classifier.

Original: Specifically, given an input image, its attribution map highlights important features that heavilyinfluence the classifier’s decision.
Revised: Specifically, given an input image, its attribution map highlights important features that heavily influence the classifier’s decision.

Original: A straightforward approach is to calculate the gradient of the
Revised: A straightforward approach is to calculate the gradient of the

Original: DNN model’s output with respect to each input pixel [20], which we refer to as Grad.
Revised: DNN model’s output with respect to each input pixel [24], which we refer to as Grad.

Original: Grad thusproduces an attribution map of which the size is identical to that of input images.
Revised: Grad thus produces an attribution map of which the size is identical to that of input images.

Original: The larger theabsolute Grad value of a pixel, the greater influence the corresponding pixel has in deriving a model’sdecision.
Revised: The larger the absolute Grad value of a pixel, the greater influence the corresponding pixel has in deriving a model’s decision.

Original: Numerous variants have been proposed to supplement or extend the explainability of
Revised: Numerous variants have been proposed to supplement or extend the explainability of

Original: Grad [19, 21, 22].
Revised: Grad [23, 25, 26].

Original: In this paper, we use Grad as a basic type of for model explanations.
Revised: In this paper, we use Grad as a basic type of for model explanations.

Original: The classactivation map (CAM) [28] is also a prevalent means of computing model explanations.
Revised: The class activation map (CAM) [33] is also a prevalent means of computing model explanations.

Original: It provides acoarse-grained activation map that represents the state of intermediate convolutional layers.
Revised: It provides a coarse-grained activation map that represents the state of intermediate convolutional layers.

Original: Gradientweighted CAM or Grad-CAM [17] is an extension of CAM.
Revised: Gradientweighted CAM or Grad-CAM [21] is an extension of CAM.

Original: Unlike CAM, Grad-CAM does notrequire a specific model architecture.
Revised: Unlike CAM, Grad-CAM does not require a specific model architecture.

Original: It performs partial back-propagation until reaching the finalconvolutional layer before the fully-connected (FC) layers. The activation maps are then aggregated
Revised: It performs partial back-propagation until reaching the final

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Revised: convolutional layer before the fully-connected (FC) layers.

Original:  by globally pooling all existing channels, thus constituting a heatmap, which can be overlaid upon theoriginal image.
Revised: The activation maps are then aggregated by globally pooling all existing channels, thus constituting a heatmap, which can be overlaid upon the original image.

Original: The layer-wise relevance propagation (LRP) takes a holistic approach to explaining amodel’s decision [3].
Revised: The layer-wise relevance propagation (LRP) takes a holistic approach to explaining a model’s decision [3].

Original: It performs importance decomposition by redistributing relevance scores alongthe backward pass and derives the relative contribution of each pixel of the input image.
Revised: It performs importance decomposition by redistributing relevance scores along the backward pass and derives the relative contribution of each pixel of the input image.

Original: Model Inversion Attacks.
Revised: Model Inversion Attacks.

Original: An MI attack refers to an attempt to invert a given model’s output, thuscomputing its approximate input.
Revised: An MI attack refers to an attempt to invert a given model’s output, thus computing its approximate input.

Original: The first MI attack targeted simple linear models to reconstructcertain genetic markers from medical dosage predictions [8].
Revised: The first MI attack targeted simple linear models to reconstruct certain genetic markers from medical dosage predictions [10].

Original: Fredrikson et al.
Revised: Fredrikson et al.

Original: [7] expanded this ideato target other types of machine learning (ML) algorithms, such as decision trees and shallow neuralnetworks, which enable the attack to target facial recognition models.
Revised: [9] expanded this idea to target other types of machine learning (ML) algorithms, such as decision trees and shallow neural networks, which enable the attack to target facial recognition models.

Original: These studies leveraged themaximum a posteriori principle to construct input features that maximize the likelihood of a givenmodel’s response.
Revised: These studies leveraged the maximum a posteriori principle to construct input features that maximize the likelihood of a given model’s response.

Original: However, their reconstructed images were blurry and of low-resolution; thesemethods are unfit for reconstructing input images against complicated DNN models.
Revised: However, their reconstructed images were blurry and of low-resolution; these methods are unfit for reconstructing input images against complicated DNN models.

Original: Yang et al.
Revised: Yang et al.

Original: [25] proposed the idea of utilizing a dedicated DNN model to perform MI attacks.
Revised: [29] proposed the idea of utilizing a dedicated DNN model to perform MI attacks.

Original: Theypretrained this inversion model with inputs that the attacker generates and their corresponding outputsfrom a target DNN model.
Revised: They pretrained this inversion model with inputs that the attacker generates and their corresponding outputs from a target DNN model.

Original: This training-based MI attack drastically improved the model inversionperformance.
Revised: This training-based MI attack drastically improved the model inversion performance.

Original: Follow-up works have demonstrated the use of generative adversarial networks (GANs)to improve the inversion performance [2, 4, 23, 26].
Revised: Follow-up works have demonstrated the use of generative adversarial networks (GANs) to improve the inversion performance [2, 5, 27, 31].

Original: Recently, Zhao et al.
Revised: Recently, Zhao et al.

Original: [27] showed that theadditional information leakage from XAI models can be further exploited to enhance the effectivenessof the MI attack.
Revised: [32] showed that the additional information leakage from XAI models can be further exploited to enhance the effectiveness of the MI attack.

Original: While previous GAN-based attacks focus on producing generic images of a targetclass, the explanation-aware attack by Zhao et al.
Revised: While previous GAN-based attacks focus on producing generic images of a target class, the explanation-aware attack by Zhao et al.

Original: [27] is able to reconstruct features specific to testinstances, such as face images with hats, glasses, or earrings.
Revised: [32] is able to reconstruct features specific to test instances, such as face images with hats, glasses, or earrings.

Original: However, to our knowledge, no defensemechanism has been proposed to defend against such explanation-aware MI attacks.
Revised: However, to our knowledge, no defense mechanism has been proposed to defend against such explanation-aware MI attacks.

Original:  Explanation-aware Model Inversion Attack
Revised:  Explanation-aware Model Inversion Attack

Original: Previous studies have demonstrated the critical threat that generative MI attacks impose.
Revised: Previous studies have demonstrated the critical threat that generative MI attacks impose.

Original: In thissection, we describe the threat model for conducting state-of-the-art generative MI attacks againstwhich GNIME is designed to protect model explanations.
Revised: In this section, we describe the threat model for conducting state-of-the-art generative MI attacks against which GNIME is designed to protect model explanations.

Original: We then explain the state-of-the-artexplanation-aware MI attacks using generative networks along with notations.
Revised: We then explain the state-of-the-art explanation-aware MI attacks using generative networks along with notations.

Original: 3.1 Threat Model
Revised: 3.1 Threat Model

Original: We consider a target classifier f providing black-box access whereby the adversary can query fand obtain its output ˆ y in the form of a confidence vector.
Revised: We consider a target classifier f providing black-box access whereby the adversary can query f and obtain its output ˆ y in the form of a prediction vector.

Original: Each vector value indicates the predictedprobability for the corresponding class.
Revised: Each vector value indicates the predicted probability for the corresponding class.

Original: Furthermore, the target service leveraging f also providesa model explanation e (e.g., a Grad, Grad-CAM, or LRP saliency map) that illustrates importantfeatures contributing to f emitting ˆ y .
Revised: Furthermore, the target service leveraging f also provides a model explanation e (e.g., a Grad, Grad-CAM, or LRP saliency map) that illustrates important features contributing to f emitting ˆ y .

Original: We assume a man-in-the-middle adversary who observes a pair of e and ˆ y for an input imageinaccessible to the adversary.
Revised: We assume a man-in-the-middle adversary who observes a pair of e and ˆ y for an input image inaccessible to the adversary.

Original: Thus, the adversary’s goal is to reconstruct a high-fidelity input imagegiven e and ˆ y .
Revised: Thus, the adversary’s goal is to reconstruct a high-fidelity input image given e and ˆ y .

Original: Furthermore, we assume that the adversary is able to obtain an auxiliary dataset D auxwhose underlying data distribution is similar to that of the training data D for the target classifier f .
Revised: Furthermore, we assume that the adversary is able to obtain an auxiliary dataset D aux whose underlying data distribution is similar to that of the training data D for the target classifier f .

Original: The adversary leverages this auxiliary dataset to train her inversion networks f A .
Revised: The adversary leverages this auxiliary dataset to train her inversion networks f A .

Original: Since the adversarycan collect (ˆ y, e ) by querying f for each input x ∈ D aux , she exploits both ˆ y and e to train f A .
Revised: Since the adversary can collect (ˆ y, e ) by querying f for each input x ∈ D aux , she exploits both ˆ y and e to train f A .

Original: Note that our adversary model is analogous to that of Zhao et al.
Revised: Note that our adversary model is analogous to that of Zhao et al.

Original: [27], which constitutes a strongblack-box adversary model.
Revised: [32], which constitutes a strong black-box adversary model.

Original: Also, Yang et al.
Revised: Also, Yang et al.

Original: [25] assumes a similar man-in-the-middle adversarywho reconstructs input images from prediction vectors observed from f .
Revised: [29] assumes a similar man-in-the-middle adversary who reconstructs input images from prediction vectors observed from f .

Original: Our goal is to protect themodel explanations from this strong black-box adversary conducting MI attacks.
Revised: Our goal is to protect the model explanations from this strong black-box adversary conducting MI attacks.

Original: 3.2 Model Inversion Attacks Using Model Explanations
Revised: 3.2 Model Inversion Attacks Using Model Explanations

Original: Zhao et al.
Revised: Zhao et al.

Original: [27] proposed a novel black-box MI attack (ExpMI) that exploits prediction vectorsgenerated from a target classifier f as well as the model explanations for each decision.
Revised: [32] proposed a novel black-box MI attack (ExpMI) that exploits prediction vectors generated from a target classifier f as well as the model explanations for each decision.

Original: To reconstructthe input images, they trained a transposed CNN model f A based on auxiliary data, each of whichwas a prediction vector paired with its model explanation.
Revised: To reconstruct the input images, they trained a transposed CNN model f A based on auxiliary data, each of which was a prediction vector paired with its model explanation.

Original: Consider a target classifier f
Revised: Consider a target classifier f

Original: that is a function f ( x ) → ( e, ˆ y ) that emits a prediction vector andmodel explanation (i.e., an attribution map).
Revised: that is a function f ( x ) → ( e, ˆ y ) that emits a prediction vector and model explanation (i.e., an attribution map).

Original: The adversary’s goal is to compute f A ( e, ˆ y )
Revised: The adversary’s goal is to compute f A ( e, ˆ y )

Original: → ˆ x , whichtakes explanation e and ˆ y to produce ˆ x , revealing sensitive attributes of x .
Revised: → ˆ x , which takes explanation e and ˆ y to produce ˆ x , revealing sensitive attributes of x .

Original: Figure 3 illustrates asimplified structure of the ExpMI model.
Revised: Figure 3 illustrates a simplified structure of the ExpMI model.

Original: To take advantage of structural information in activationmaps, ExpMI utilizes a U-Net-style bypass connection along with flattened inputs directly appendedto the intermediate embedding.[Equation]
Revised: To take advantage of structural information in activation maps, ExpMI utilizes a U-Net-style bypass connection along with flattened inputs directly appended to the intermediate embedding.

Original: [Table]
Revised: [Table]

Revised: _base64="Q5jnLNqgSmQAnb22w0hnW60w3wA=">AAAB7nicbZC7SgNBFIbPxluMt6ilzeIiWIVdi2gZsNEugpsEkhBmJ7PJkNnZZeassCxpfAMbC0VsLX0JX8DOx/ANnFwKTfxh4OP/z2HOOUEiuEbX/bIKK6tr6xvFzdLW9s7uXnn/oKHjVFHm01jEqhUQzQSXzEeOgrUSxUgUCNYMRpeTvHnHlOaxvMUsYd2IDCQPOSVorGZnSDDPxr2y41bcqexl8Obg1Jzvj3v/vVrvlT87/ZimEZNIBdG67bkJdnOikFPBxqVOqllC6IgMWNugJBHT3Xw67tg+MU7fDmNlnkR76v7uyEmkdRYFpjIiONSL2cT8L2unGF50cy6TFJmks4/CVNgY25Pd7T5XjKLIDBCquJnVpkOiCEVzoZI5gre48jI0zipeteLdeE7tGmYqwhEcwyl4cA41uII6+EBhBA/wBM9WYj1aL9brrLRgzXsO4Y+stx/eeJOq</latexit>

Original: [Figure]
Revised: [Figure]

Original: The adversary takes advantage of e , which leakssensitive information pertaining to x and drastically improves the reconstruction performanceof f A .
Revised: The adversary takes advantage of e , which leaks sensitive information pertaining to x and drastically improves the reconstruction performance of f A .

Original: As Figure 1 shows, the quality of thereconstructed image ˆ x with e is significantly better than the MI attack using prediction vectorsalone generates.
Revised: As Figure 1 shows, the quality of the reconstructed image ˆ x with e is significantly better than the MI attack using prediction vectors alone generates.

Original: Considering that model explanation has become instrumental in identifying the reasons behindthe decisions of DNN models in various industry sectors, it becomes inevitable for corporationsto provide model explanations.
Revised: Considering that model explanation has become instrumental in identifying the reasons behind the decisions of DNN models in various industry sectors, it becomes inevitable for corporations to provide model explanations.

Original: However, providing such information also jeopardizes privacy byrevealing sensitive attributes of reconstructed images.
Revised: However, providing such information also jeopardizes privacy by revealing sensitive attributes of reconstructed images.

Original: To this end, we propose GNIME, a noveldefense framework that learns to inject noise into model explanations, thus minimizing the privacyrisk of ExpMI attacks and maximizing the utility of noise-injected model explanations.
Revised: To this end, we propose GNIME, a novel defense framework that learns to inject noise into model explanations, thus minimizing the privacy risk of ExpMI attacks and maximizing the utility of noise-injected model explanations.

Original:  Learning to Generate Noise for Model Explanations
Revised:  Learning to Generate Noise for Model Explanations

Original: This section introduces our defense framework, GNIME.

Original: Figure 2 depicts the overall workflow of GNIME.
Revised: Figure 2 depicts the overall workflow of GNIME.

Original: Phase I trains the noise injector and MI attack networks, thus inducing the noise injectornetwork to inject inversion-resistant perturbations into model explanations.
Revised: Phase I trains the noise injector and MI attack networks, thus inducing the noise injector network to inject inversion-resistant perturbations into model explanations.

Original: Phase II then deploys thenoise injector network in injecting noise into each model explanation upon its release.
Revised: Phase II then deploys the noise injector network in injecting noise into each model explanation upon its release.

Original: 4.1 Training GNIME

Original: Given a target classifier f ( x ) → ( e, ˆ y ) , GNIME learns to inject noise into e to minimize the MIthreat.
Revised: Given a target classifier f ( x ) → ( e, ˆ y ) , GNIME learns to inject noise into e to minimize the MI threat.

Original: Our approach utilizes the noise generator network ( f NG ) and inversion network ( f INV ), bothof which are trained upon D T
Revised: Our approach utilizes the noise generator network ( f NG ) and inversion network ( f INV ), both of which are trained upon D T

Original: = { x i , y i } Ni =1 , the original training data of f .
Revised: = { x i , y i } Ni =1 , the original training data of f .

Original: Let f NG ( e, ˆ y )
Revised: Let f NG ( e, ˆ y )

Original: → ˆ e be the NG network that generates a new explanation attribution map ˆ e , whichincludes noise.
Revised: → ˆ e be the NG network that generates a new explanation attribution map ˆ e , which includes noise.

Original: For the INV network, we design f INV ( e, ˆ y )
Revised: For the INV network, we design f INV ( e, ˆ y )

Original: → ˆ x that takes e and ˆ y , then computesan inversion image ˆ x .
Revised: → ˆ x that takes e and ˆ y , then computes an inversion image ˆ x .

Original: For the loss functions of f NG and f INV , we use an explanation distortion loss( L xd ) to encourage minimum perturbation to the explanations and an image reconstruction loss ( L re )to encourage inversion performance.
Revised: For the loss functions of f NG and f INV , we use an explanation distortion loss ( L xd ) to encourage minimum perturbation to the explanations and an image reconstruction loss ( L re ) to encourage inversion performance.

Original: We use the pixel-wise mean squared error (MSE) between e andˆ e for L xd , and the MSE between x and ˆ x for L re .
Revised: We use the pixel-wise mean squared error (MSE) between e and ˆ e for L xd , and the MSE between x and ˆ x for L re .

Original: Inspired by Generative Adversarial Networks [9], we form a two-player minimax game in whichf NG and f INV are trained together using two different objectives.
Revised: Inspired by Generative Adversarial Networks [11], we form a two-player minimax game in which f NG and f INV are trained together using two different objectives.

Original: Specifically, the training phaseconsists of two steps for each epoch.
Revised: Specifically, the training phase consists of two steps for each epoch.

Original: In the first step, we freeze f NG and train f INV by minimizing the following loss:
Revised: In the first step, we freeze f NG and train f INV by minimizing the following loss:

Original: [Equation]
Revised: [Equation]

Original: The objective is to improve the inversion capability of f INV given ( e, ˆ y ) and (ˆ e, ˆ y ) .
Revised: The objective is to improve the inversion capability of f INV given ( e, ˆ y ) and (ˆ e, ˆ y ) .

Original: Note that L INVincludes the reconstruction loss from the clean explanations in order to prevent the network fromoverfitting to the perturbed explanations.
Revised: Note that L INV includes the reconstruction loss from the clean explanations in order to prevent the network from overfitting to the perturbed explanations.

Original: For the second step, we freeze f INV and train f NG byminimizing the following loss:
Revised: For the second step, we freeze f INV and train f NG by minimizing the following loss:

Original: [Equation]
Revised: [Equation]

Original: Input: target classifier f input data X x

Original: } Output: noise generator network f NG Generate Y from to for doSample a batch of m : {
Revised: do

Original: , consisting of pairs

Original: X Y and E Let ˆ ei = f NG ( e , ˆ y i ) in
Revised: Let ˆ e

Original: Update f f θ fNG 1 m

Original: =

Original: It aims to minimize the distortion between e and ˆ e , while maximizing the error between x and ˆ x .
Revised: It aims to minimize the distortion between e and ˆ e , while maximizing the error between x and ˆ x .

Original: Therefore, we train f NG to generate a noise-injected attribution map that is similar to the originalattribution map but causes f INV to generate an inversion image dissimilar from its original image.
Revised: Therefore, we train f NG to generate a noise-injected attribution map that is similar to the original attribution map but causes f INV to generate an inversion image dissimilar from its original image.

Original: Algorithm 1 explains our training process.
Revised: Algorithm 1 explains our training process.

Original: In L NG , we set λ = 500 for CelebA models and λ = 100for MNIST and CIFAR-10 models, then deploy the final model after 500 epochs.
Revised: In L NG , we set λ = 500 for CelebA models and λ =for MNIST and CIFAR-10 models, then deploy the final model after 500 epochs.

Original: In the worst case, f NG will merely cause a regularization effect on f INV , rendering it robust to anynoisy explanation.
Revised: In the worst case, f NG will merely cause a regularization effect on f INV , rendering it robust to any noisy explanation.

Original: However, as f NG is trained over epochs, it learns to capture and suppress featuresinside explanations that contain exploitable information regarding the original input image.
Revised: However, as f NG is trained over epochs, it learns to capture and suppress features inside explanations that contain exploitable information regarding the original input image.

Original: Once thetraining is finished, we discard f INV and only make use of f NG for Phase II.
Revised: Once the training is finished, we discard f INV and only make use of f NG for Phase II.

Original: For every attributionmap e , we use f NG to generate a noise-injected attribution map ˆ e , then release ˆ e instead of e upon agiven query x .
Revised: For every attribution map e , we use f NG to generate a noise-injected attribution map ˆ e , then release ˆ e instead of e upon a given query x .

Original: Additionally, GNIME service providers can inject less noise into the explanations with an arbitrary
Revised: Additionally, GNIME service providers can inject less noise into the explanations with an arbitrary

Original: MSE bound, γ .
Revised: MSE bound, γ .

Original: In this case, GNIME computes a new attribution map e ′
Revised: In this case, GNIME computes a new attribution map e ′

Original: (see Equation 3); wenormalize the noise difference n between ˆ e and e by dividing it by the square root of the variance ofn over γ and then add this normalized difference to e :
Revised: (see Equation 3); we normalize the noise difference n between ˆ e and e by dividing it by the square root of the variance of n over γ and then add this normalized difference to e :

Original: [Equation]
Revised: [Equation]

Original: n
Revised: n

Revised: [Equation]

Revised: e

Original: [Equation]
Revised: [Equation]

Original:  σ 2 ( n ) denotes the pixel variance of n .
Revised: ) σ 2 ( n ) denotes the pixel variance of n .

Original: Therefore, by controlling γ , GNIME service providers canutilize the trade-off and choose to preserve better explanation functionality in exchange for defensivecapability.
Revised: Therefore, by controlling γ , GNIME service providers can utilize the trade-off and choose to preserve better explanation functionality in exchange for defensive capability.

Original: Refer to the supplementary material for further analysis regarding the trade-off.
Revised: Refer to the supplementary material for further analysis regarding the trade-off.

Original:  Experiments 
Revised:  Experiments 5.1 Experimental Setup

Original: 5.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: We evaluate GNIME using three datasets: (1) CelebA [13], (2) MNIST [11], and (3) CIFAR-10 [10], each of which is freely available fora research purpose.
Revised: We evaluate GNIME using four datasets: (1) CelebA [17], (2) MNIST [14], (3) CIFAR10 [13], and (4) ImageNet [6] , each of which is freely available for research purposes.

Original: For CelebA, we use itsbalanced subset consisting of 30,000 face images of 1000 individuals.
Revised: For CelebA, we use its balanced subset consisting of 30,000 face images of 1000 individuals.

Original: To obtain a tighter boundfor the faces, we crop the center and resize each CelebA image to 128 × 128.
Revised: To obtain a tighter bound for the faces, we crop the center and resize each CelebA image to 128 × 128.

Original: MNIST consists of70,000 images of handwritten digits, and CIFAR-10 has 60,000 images of 10 different objects.
Revised: MNIST consists of 70,000 images of handwritten digits, and CIFAR-10 has 60,000 images of 10 different objects.

Original: Both
Revised: Both

Original: MNIST and CIFAR-10 images were resized to 32 × 32.
Revised: MNIST and CIFAR-10 images were resized to 32 × 32.

Original: We use the same datasetsadopted by previousstudies [25–27] to demonstrate how effectively GNIME mitigates the MI threat.
Revised: We note that previous studies [29, 31, 32] adopted the same datasets to demonstrate how effectively their frameworks mitigate the MI threat.

Revised: In addition, we evaluate GNIME on ImageNet-100 [6], which is a large scale dataset of higher resolution, consisting of 100 classes randomly selected classes from ImageNet; each image is resized to 256 × 256.

Original: We split each dataset into two disjoint sets of the same size: the target and attack datasets.
Revised: We split each dataset into two disjoint sets of the same size: the target and attack datasets.

Original: The targetdataset is used to train a target model f and GNIME.
Revised: The target dataset is used to train a target model f and GNIME.

Original: The attack dataset is used to train the inversionnetworks for explanation-aware MI attacks and to test GNIME.
Revised: The attack dataset is used to train the inversion networks for explanation-aware MI attacks and to test GNIME.

Original: Specifically, we split this attackdataset with an 80/20 ratio for the train/test split.
Revised: Specifically, we split this attack dataset with an 80/20 ratio for the train/test split.

Original: Note that we assume an adversary capable of training f
Revised: Note that we assume an adversary capable of training f

Original: A upon a dataset of which the underlyingdistribution is nearly identical to that of the target network’s training data.
Revised: A upon a dataset of which the underlying distribution is nearly identical to that of the target network’s training data.

Original: We intentionally assumethis strong adversary to demonstrate how good GNIME is in mitigating the MI threat.
Revised: We intentionally assume this strong adversary to demonstrate how good GNIME is in mitigating the MI threat.

Original: Moreover,we also assess the performance of GNIME against a weaker but more practical adversary usingan auxiliary dataset that has a different underlying distribution (i.e., a separately collected set forconducting a similar task) 
Revised: Moreover, we also assess the performance of GNIME against a weaker but more practical adversary using an auxiliary dataset that has a different underlying distribution (i.e., a separately collected set for conducting a similar task) [29].

Original: For instance, when attacking the CelebA model, we randomly sampleface images from FaceScrub [16] to train f A .
Revised: For instance, when attacking the CelebA model, we randomly sample face images from FaceScrub [20] to train f A .

Original: Note that the overlapping identities from FaceScrubwhich also appear in CelebA are removed before the sampling process.
Revised: Note that the overlapping identities from FaceScrub which also appear in CelebA are removed before the sampling process.

Original: We extract a tighter bound ofeach FaceScrub face according to its official bounding box information.[Table] [Table]
Revised: We extract a tighter bound of each FaceScrub face according to its official bounding box information.

Original: Target models.
Revised: Target Models.

Original: We implementtwo different target models.
Revised: We implement different target models for different datasets.

Original: For CelebA, the target model consists ofthree convolutional layers followed by two fully-connected (FC) layers.
Revised: For CelebA, the target model consists of three convolutional layers followed by two fully-connected (FC) layers.

Original: We employ dropout beforeeach FC layer to avoid overfitting.
Revised: We employ dropout before each FC layer to avoid overfitting.

Original: For MNIST and CIFAR-10, we use a target model with twoconvolutional layers followed by two FC layers.
Revised: For MNIST and CIFAR-10, we use a target model with two convolutional layers followed by two FC layers.

Original: We selected the same models that Zhao et al.
Revised: We selected the same models that Zhao et al.

Original: [27]used to demonstrate their MI attacks.
Revised: [32] used to demonstrate their MI attacks.

Revised: For ImageNet-100, we used the model from

Revised: CelebA with an additional convolutional layer.

Original: Inversion methods.
Revised: Inversion Methods.

Original: We use two MI attack methods (ExpMI and PredMI) as baselines in ourexperiments.
Revised: We use two MI attack methods (ExpMI and PredMI) as baselines in our experiments.

Original: Specifically, we compare the efficacy of various defense mechanisms against the stateof-the-art ExpMI attack [27].
Revised: Specifically, we compare the efficacy of various defense mechanisms against the stateof-the-art ExpMI attack [32].

Original: In addition, we consider the prediction-only MI attack using no modelexplanations (PredMI), suggested by Yang et al.
Revised: In addition, we consider the prediction-only MI attack using no model explanations (PredMI), suggested by Yang et al.

Original: Note that the inversion results via PredMIform an upper bound that represents the ideal case for any defenses in which model explanationsleak no additional knowledge that the adversary is able to exploit.
Revised: Note that the inversion results via PredMI form an upper bound that represents the ideal case for any defenses in which model explanations leak no additional knowledge that the adversary is able to exploit.

Original: We conducted each MI attack fivetimes and reported average metrics with standard deviations.
Revised: We conducted each MI attack five times and reported average metrics with standard deviations.

Original: All experiments took place on a systemequipped with 512GBs of RAM, two Intel Xeon Gold 6258R CPUs, and four RTX 3090 GPUs.
Revised: All experiments took place on a system equipped with 512GBs of RAM, two Intel Xeon Gold 6258R CPUs, and four RTX 3090 GPUs.

Original: 5.2 Evaluation Metrics
Revised: 5.2 Evaluation Metrics

Original: We assess whether the reconstructed image is close to its original image by leveraging four metrics:
Revised: We assess whether the reconstructed image is close to its original image by leveraging four metrics:

Original: MSE, SSIM, TCA, and DeePSiM, which have been extensively used for quantitative evaluations inprevious studies [25–27].
Revised: MSE, SSIM, TCA, and DeePSiM, which have been extensively used for quantitative evaluations in previous studies [29, 31, 32].

Original: Mean Squared Error (MSE).
Revised: Mean Squared Error (MSE).

Original: The MSE is a metric that measures the distance between a reconstructed image and its corresponding original image.
Revised: The MSE is a metric that measures the distance between a reconstructed image and its corresponding original image.

Original: A small MSE indicates that the reconstructedimage is close to its original image.
Revised: A small MSE indicates that the reconstructed image is close to its original image.

Original: For normalization, input pixel values of the inputs are scaled tobe within the range of [0 , 1] .
Revised: For normalization, input pixel values of the inputs are scaled to be within the range of [0 , 1] .

Original: Structural Similarity Index Measure (SSIM).
Revised: Structural Similarity Index Measure (SSIM).

Original: The SSIM computes the distortion of structuralinformation between two images [24].
Revised: The SSIM computes the distortion of structural information between two images [28].

Original: Since humans are good at recognizing structural differencesbetween two images, this metric shows human perceptual similarity between two images.
Revised: Since humans are good at recognizing structural differences between two images, this metric shows human perceptual similarity between two images.

Original: An SSIMscore spans from 0 to 1, and the higher score indicates better reconstruction.
Revised: An SSIM score spans from 0 to 1, and the higher score indicates better reconstruction.

Original: Deep Perceptual Similarity Metric (DeePSiM).
Revised: Deep Perceptual Similarity Metric (DeePSiM).

Original: The DeePSiM shows the similarity betweentwo images at an intermediate representation level [5].
Revised: The DeePSiM shows the similarity between two images at an intermediate representation level [7].

Original: Unlike SSIM, which illustrates perceptualsimilarity, DeePSiM explains perceptual similarity from the neural network perspective.
Revised: Unlike SSIM, which illustrates perceptual similarity, DeePSiM explains perceptual similarity from the neural network perspective.

Original: Specifically,we trained a separate evaluation classifier f E , which shares the same architecture and training datasetwith the target model.
Revised: Specifically, we trained a separate evaluation classifier f E , which shares the same architecture and training dataset with the target model.

Original: We then computed the feature distance between intermediate results from thepenultimate layer of f E , comparing the high-level features identified by f E [27].
Revised: We then computed the feature distance between intermediate results from the penultimate layer of f E , comparing the high-level features identified by f E [32].

Original: If f INV accuratelyreconstructs the high-level image features, it results in a high DeePSiM score.
Revised: If f INV accurately reconstructs the high-level image features, it results in a high DeePSiM score.

Original: Transferable Classification Accuracy (TCA).
Revised: Transferable Classification Accuracy (TCA).

Original: We also measure a TCA to evaluate whether invertedimages are generally recognizable by another DNN classifier of which the structure is the same as thatof the target model.
Revised: We also measure a TCA to evaluate whether inverted images are generally recognizable by another DNN classifier of which the structure is the same as that of the target model.

Original: This metric indirectly shows whether the reconstructed image incorporates thekey features of its original image.
Revised: This metric indirectly shows whether the reconstructed image incorporates the key features of its original image.

Original: We measured this metric using f E .
Revised: We measured this metric using f E .

Original: Note that f E would correctlypredict the reconstructed image if the reconstructed image successfully incorporates the key featuresof the original image, rather than to overfit to unimportant details captured only by the target classifier.
Revised: Note that f E would correctly predict the reconstructed image if the reconstructed image successfully incorporates the key features of the original image, rather than to overfit to unimportant details captured only by the target classifier.

Original: 5.3 Experiment Results 
Revised: 5.3 Experiment Results 5.3.1 Defending against MI Attacks

Original: 5.3.1 Defending against MI Attacks

Original: We evaluate the performance of vanilla explanation-aware MI attacks and then measure the efficacyof GNIME in decreasing information leakage from such MI attacks.
Revised: We evaluate the performance of vanilla explanation-aware MI attacks and then measure the efficacy of GNIME in decreasing information leakage from such MI attacks.

Original: Specifically, we compare fivesettings (two baseline attacks and three defenses against ExpMI attacks): (1) MI attacks using themodel’s prediction outputs alone (PredMI); (2) MI attacks using both prediction outputs and modelexplanations (ExpMI); (3) ExpMI attacks against a defense that injects Gaussian noise into saliencymaps (RND); (4) ExpMI attacks against a defense that injects optimized noise into saliency maps(OND); and (5) ExpMI attacks against GNIME.
Revised: Specifically, we compare five settings (two baseline attacks and three defenses against ExpMI attacks): (1) MI attacks using the model’s prediction outputs alone (PredMI); (2) MI attacks using both prediction outputs and model explanations (ExpMI); (3) ExpMI attacks against a defense that injects Gaussian noise into saliency maps (RND); (4) ExpMI attacks against a defense that injects optimized noise into saliency maps [Table] C I

Original:  When measuring the defensive capability of RND,
Revised: F AR - [Table] (OND); and (5) ExpMI attacks against GNIME. When measuring the defensive capability of RND,

Original: OND, and GNIME, each MI network was separately trained using the explanations perturbed by thecorresponding defence method.
Revised: OND, and GNIME, each MI network was separately trained using the explanations perturbed by the corresponding defense method.

Original: This resembles the real-world adversary in our black-box setting,who has no way of obtaining clean model explanations.
Revised: This resembles the real-world adversary in our black-box setting, who has no way of obtaining clean model explanations.

Original: Table 1 compares the performance of GNIME against these experimental settings assuming a target
Revised: Table 1 compares the performance of GNIME against these experimental settings assuming a target

Original: ML service providing Grad, Grad-CAM, and LRP, respectively.
Revised: ML service providing Grad, Grad-CAM, and LRP, respectively.

Original: Recall from §3 that we assume astrong adversary in possession of auxiliary data of which the distribution is the same as the trainingdataset of the target model.
Revised: Recall from §3 that we assume a strong adversary in possession of auxiliary data of which the distribution is the same as the training dataset of the target model.

Original: We thus use the train split of our attack dataset for training GNIME andthe test split for measuring the metrics (§5.1).
Revised: We thus use the train split of our attack dataset for training GNIME and the test split for measuring the metrics (§5.1).

Original: Considering that ExpMI shows better reconstruction quality compared to PredMI in all the cases,exploiting model explanations to perform MI attacks indeed contributes to additional informationleakage.
Revised: Considering that ExpMI shows better reconstruction quality compared to PredMI in all the cases, exploiting model explanations to perform MI attacks indeed contributes to additional information leakage.

Original: Surprisingly, the SSIM score significantly boosted from 0.1688 to 0.8862 when targeting CIFAR-10-LRP.
Revised: Surprisingly, the SSIM score significantly boosted from 0.1688 to 0.8862 when targeting CIFAR-10-LRP.

Original: These results demonstrate the necessity of defense against ExpMI attacks.
Revised: These results demonstrate the necessity of defense against ExpMI attacks.

Original: We now compare the performance of GNIME against other two defense settings (RND and OND) thatadd noise to achieve the same defensive goal.
Revised: We now compare the performance of GNIME against other two defense settings (RND and OND) that add noise to achieve the same defensive goal.

Original: RND corresponds to a naive approach that randomlyadds Gaussian noise, which serves as our defense baseline.
Revised: RND corresponds to a naive approach that randomly adds Gaussian noise, which serves as our defense baseline.

Original: For OND, instead of training f NG , we usethe projected gradient descent [14] to optimize the input image towards increasing the loss functionof f INV .
Revised: For OND, instead of training f NG , we use the projected gradient descent [18] to optimize the input image towards increasing the loss function of f INV .

Original: Note that adding huge noise can prevent information leakage, but it would underminethe ability to explain the model’s decisions.
Revised: Note that adding huge noise can prevent information leakage, but it would undermine the ability to explain the model’s decisions.

Original: Hence, for fair comparison, we clip the perturbationmagnitude of RND and OND to align with the average perturbation size of GNIME.
Revised: Hence, for fair comparison, we clip the perturbation magnitude of RND and OND to align with the average perturbation size of GNIME.

Original: As shown in the table, GNIME succeeds in decreasing the information leakage threat for all theexplanations and data types.

Original: Although RND also slightly alleviates the MI threat by degrading theinversion quality, GNIME significantly outperforms RND in all the cases.

Original: Furthermore, GNIMEshows superior performance compared to OND as well in all cases.

Original: For example, when targeting

Original: CelebA-LRP, RND and OND lower the TCA by 0.0588 and 0.0940, respectively.

Original: On the other hand,

Original: GNIME decreases the TCA by 0.1821, which corresponds to 3.10 × of RND and 1.94 × of OND.

Original: From these observations, we conclude that noise optimized against the f INV fails to generalize onf A , which is independently trained for ExpMI attacks, and it requires a dedicated model f NG togenerate effective inversion-resistant noise.

Original: PredMI serves as an upper bound for all the presented defense frameworks, which shows the case whenthere exists no additional information leakage threat that stems from providing model explanations(§5.1).

Original: Considering this upper bound shown in the table, we believe that the degree of GNIME

Original: Target [Figure] w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: (a) ExpMI against Grad
Revised: (a) ExpMI against Grad

Original: Target Image w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: [Figure]

Original:  ExpMI against Grad-CAM
Revised: (b) ExpMI against Grad-CAM

Original: Target Image w/o defense
Revised: Target Image w/o defense

Original: RND
Revised: RND

Original: OND
Revised: OND

Original: GNIME 0.
Revised: GNIME 0.

Original: (c) ExpMI against LRP
Revised: (c) ExpMI against LRP

Original: [Figure]
Revised: As shown in the table, GNIME succeeds in decreasing the information leakage threat for all the explanations and data types. Although RND also slightly alleviates the MI threat by degrading the inversion quality, GNIME significantly outperforms RND in all the cases. Furthermore, GNIME shows superior performance compared to OND as well in all cases. For example, when targeting

Original: [Table]
Revised: CelebA-LRP, RND and OND lower the TCA by 0.0588 and 0.0940, respectively.

Original: [Table]
Revised: On the other hand, GNIME decreases the TCA by 0.1821, which corresponds to 3.10 × of RND and 1.94 × of OND. From these observations, we conclude that noise optimized against the f INV fails to generalize on f A , which is independently trained for ExpMI attacks, and it requires a dedicated model f NG to generate effective inversion-resistant noise. PredMI serves as an upper bound for all the presented defense frameworks, which shows the case when there exists no additional information leakage threat that stems from providing model explanations (§5.1).

Original:  achieving its defensive goal is close to its upper bound.
Revised: Considering this upper bound shown in the table, we believe that the degree of GNIME achieving its defensive goal is close to its upper bound.

Original: For instance, the DeePSiM score against
Revised: For instance, the DeePSiM score against

Original: PredMI and ExpMI targeting CelebA-Grad with GNIME achieves 0.1851 and 0.1880, respectively.
Revised: PredMI and ExpMI targeting CelebA-Grad with GNIME achieves 0.1851 and 0.1880, respectively.

Original: Figure 4 presents the qualitative experimental results.
Revised: Figure 4 presents the qualitative experimental results.

Original: We observe that GNIME outperforms both
Revised: We observe that GNIME outperforms both

Original: RND and OND in decreasing the reconstruction quality.
Revised: RND and OND in decreasing the reconstruction quality.

Original: When comparing all columns betweenthe third and fifth for each explanation type, GNIME clearly contributes to degrading the qualityof reconstructed images; ExpMI still reconstructs most features in the inversion images when theexplanations are under the protection of RND or OND, contrary to the inversion images with GNIME.
Revised: When comparing all columns between the third and fifth for each explanation type, GNIME clearly contributes to degrading the quality of reconstructed images; ExpMI still reconstructs most features in the inversion images when the explanations are under the protection of RND or OND, contrary to the inversion images with GNIME.

Original: We further compare the inversion results with a more practical adversary.
Revised: We further compare the inversion results with a more practical adversary.

Original: Here, the adversary cannottrain its inversion network f A with the same dataset as the target model (CelebA).
Revised: Here, the adversary cannot train its inversion network f A with the same dataset as the target model (CelebA).

Original: Instead, it drawsthe training data from another generic distribution.
Revised: Instead, it draws the training data from another generic distribution.

Original: Specifically, we use a subset of FaceScrub to trainf A .
Revised: Specifically, we use a subset of FaceScrub to train f A .

Original: This auxiliary training set is arguably easier to obtain than the original dataset [25].
Revised: This auxiliary training set is arguably easier to obtain than the original dataset [29].

Original: Table 2 summarizes the ExpMI attack performance with and without GNIME.
Revised: Table 2 summarizes the ExpMI attack performance with and without GNIME.

Original: The metrics consistentlyreport lower reconstruction performance even when the adversary leverages FaceScrub as the auxiliarydataset.
Revised: The metrics consistently report lower reconstruction performance even when the adversary leverages FaceScrub as the auxiliary dataset.

Original: Interestingly, the defensive capability of GNIME was even greater in the practical setting(e.g., using FaceScrub to attack CelebA-LRP, the degradation of TCA due to GNIME increased from84.82% degradation (0.2147 → 0.0326) to 99.43% (0.2086 → 0.0012)).
Revised: Interestingly, the defensive capability of GNIME was even greater in the practical setting (e.g., using FaceScrub to attack CelebA-LRP, the degradation of TCA due to GNIME increased from 84.82% degradation (0.2147 → 0.0326) to 99.43% (0.2086 → 0.0012)).

Original: These results demonstrate that
Revised: These results demonstrate that

Original: GNIME is more effective against the practical adversary of conducting ExpMI attacks.
Revised: GNIME is more effective against the practical adversary of conducting ExpMI attacks.

Original: 5.3.2 Preserving XAI Functionality
Revised: 5.3.2 Preserving XAI Functionality

Original: In this section, we further evaluate whether an attribution map ˆ e noised by GNIME still providesinterpretable explanations on the model’s decisions, thus preserving its functionality.
Revised: In this section, we further evaluate whether an attribution map ˆ e noised by GNIME still provides interpretable explanations on the model’s decisions, thus preserving its functionality.

Original: The XAImethods (e.g., Grad, Grad-CAM, and LRP) produce an attribution map that reflects the degree ofimportance for each pixel contributing to a target model emitting a decision.
Revised: The XAI methods (e.g., Grad, Grad-CAM, and LRP) produce an attribution map that reflects the degree of importance for each pixel contributing to a target model emitting a decision.

Original: Therefore, an input imagemultiplied by this model explanation ( x ⊙ e ) erodes unimportant pixels and leaves important pixelsintact.
Revised: Therefore, an input image multiplied by this model explanation ( x ⊙ e ) erodes unimportant pixels and leaves important pixels intact.

Original: That is, we can indirectly evaluate whether the original functionality of a model explanation eis preserved in its perturbed version ˆ e by comparing the pixels left in x ⊙ e and x ⊙ ˆ e using DeePSiM.
Revised: That is, we can indirectly evaluate whether the original functionality of a model explanation e is preserved in its perturbed version ˆ e by comparing the pixels left in x ⊙ e and x ⊙ ˆ e using DeePSiM.

Original: We compare the intermediate representation values in the penultimate layer of the target classifierf for each input: x ⊙ e and x ⊙ ˆ e .
Revised: We compare the intermediate representation values in the penultimate layer of the target classifier [Table] [Table]

Revised: [Figure]

Revised: f for each input: x ⊙ e and x ⊙ ˆ e .

Original: A higher DeePSiM score indicates a higher resemblance in thehigh-level features captured by the target model.
Revised: A higher DeePSiM score indicates a higher resemblance in the high-level features captured by the target model.

Original: Table 3 shows the average DeePSiM value of all x ⊙ e and x ⊙ ˆ e pairs in which each x is from the testsplit of the attack dataset.
Revised: Table 3 shows the average DeePSiM value of all x ⊙ e and x ⊙ ˆ e pairs in which each x is from the test split of the attack dataset.

Original: Thetable demonstrates that the model perceptual similarity between x ⊙ eand x ⊙ ˆ e is at least 0.6734, 0.9298,  and 0.8761 for the CelebA, MNIST,  and CIFAR-10 models,respectively.
Revised: The model perceptual similarity between x ⊙ e and x ⊙ ˆ e is at least 0.6734, 0.9298, 0.8761, and 0.9097 for the CelebA, MNIST, CIFAR-10, and ImageNet-100 models, respectively.

Original: This indicates that the  functionality ofˆ e largely remains intact.
Revised: This indicates that the original functionality of e largely remains intact.

Original: F r e qu e n cy MSE 0.
Revised: [Figure]

Original: Perturbation Magnitude.
Revised: Perturbation Magnitude.

Original: We analyze the distribution of MSEs between e and ˆ e pairs for
Revised: We analyze the distribution of MSEs between e and ˆ e pairs for

Original: CelebA depicted in figure 5.
Revised: CelebA depicted in Figure 5.

Original: The figure showstwo triples ( x , e , ˆ e ), each of which respectivelyrepresents a common case and an extreme caseof perturbation magnitude (i.e., MSE = 0.0015and MSE = 0.0055).
Revised: The figure shows two triples ( x , e , ˆ e ), each of which respectively represents a common case and an extreme case of perturbation magnitude (i.e., MSE = 0. and MSE = 0.0055).

Original: Note that even the perturbation of the extreme case is hardly perceptible.
Revised: Note that even the perturbation of the extreme case is hardly perceptible.

Original: These results suggest that GNIME appliesinfinitesimal perturbations to e such that it preserves its original functionality.
Revised: These results suggest that GNIME applies infinitesimal perturbations to e such that it preserves its original functionality.

Revised: We include further visualization of perturbed model explanations in the supplementary material.

Original: In addition, recall from §4.1 that one can furthervary γ in Equation 3 to control the perturbation magnitude.
Revised: In addition, recall from §4 that one can further vary γ in Equation 3 to control the perturbation magnitude.

Original: We observed a linear trade-off relationshipbetween the defensive capability and the explanation functionality of GNIME.
Revised: We observed a linear trade-off relationship between the defensive capability and the explanation functionality of GNIME.

Original: GNIME serviceproviders can use γ of their choice with this in mind.
Revised: GNIME service providers can use γ of their choice with this in mind.

Original: When choosing to use 0.0040 for γ , GNIMEachieves the 0.6340 DeePSiM.
Revised: When choosing to use 0.0040 for γ , GNIME achieves the 0.6340 DeePSiM.

Original: However, when limiting the perturbation size to 0.0005, the DeePSiMscore increases to 0.8497.
Revised: However, when limiting the perturbation size to 0.0005, the DeePSiM score increases to 0.8497.

Original: Please refer to the supplementary material for further experimental results.
Revised: Please refer to the supplementary material for further experimental results.

Revised: 5.3.3 Investigating Additional Information Leakage.

Revised: Although GNIME significantly lowers the risk of information leakage to the degree to that of PredMI, information leakage still remains to some extent even after its deployment.

Revised: However, we stress that

Revised: GNIME focuses on preventing additional information leakage that stems from model explanations; therefore, the inversion results of PredMI form an upper bound of our defense.

Revised: Note that GNIME is capable of obscuring several key attributes in the original images, providing inversion results similar to those of PredMI.

Original: Limitations
Revised: For instance, Figure 1 shows that GNIME removes critical features in the reconstructed images, such as sunglasses, cap, or gender information. We additionally evaluate whether GNIME is able to obscure several key attributes of reconstructed images. Similar to Chen et al. [5], we trained attribute classifiers using CelebA and evaluated whether the attribute classifiers is able to correctly predict attributes when the reconstructed images are provided.

Revised: Specifically, we trained eight attribute classifiers, employing the ResNet50V2 architecture from He et al.

Revised: [12] with its final layer replaced for binary classification.

Revised: Note that the training set used to train these classifiers is disjoint from that we used in the input reconstruction.

Revised: We report the

Revised: F1 score to measure the performance of these attribute classifiers.

Revised: Figure 6 summarizes the evaluation results.

Revised: GNIME performed best in terms of deteriorating the attribute prediction performance for all

Revised: [Figure]

Revised: attribute types.

Revised: On average, ExpMI without defense reports an F1 score of 0.7672.

Revised: Note that GNIME decreases it to 0.3048, which is close to the F1 score of PredMI.

Revised:  Discussions and Limitations

Revised: Constrained Obfuscation.

Revised: The concept of obfuscating sensitive information while retaining useful information has been explored in various fields in establishing privacy [4, 16] and fairness[30].

Revised: Their common approach is to obfuscate given input instances and then jointly train a target model to retain a high performance even for those obfuscated instances.

Revised: By constast, GNIME does not directly alter the target model paramaters and focuses only on obfuscating model explanations to minimize the MI threat.

Revised: This enables the deployment of legacy models without any retraining steps while preversing the orignal model performance.

Original: Model extraction.
Revised: Model Extraction.

Original: Zhao et al.
Revised: Zhao et al.

Original: [27] proposed an ExpMI attack even when a target DNN model doesnot provide any explanations (ExpMI-3), which we do not assume in the threat model.
Revised: [32] proposed an ExpMI attack even when a target DNN model does not provide any explanations (ExpMI-3), which we do not assume in the threat model.

Original: The core ideaof ExpMI-3 is to build a high-performing surrogate model through a successful model extraction (ME)attack and then construct surrogate explanations based on this surrogate model.
Revised: The core idea of ExpMI-3 is to build a high-performing surrogate model through a successful model extraction (ME) attack and then construct surrogate explanations based on this surrogate model.

Original: However, we stressthat once the adversary successfully constructs the surrogate model, the surrogate model exposes thetarget model to numerous white-box attacks, not only limited to the ExpMI threat.
Revised: However, we stress that once the adversary successfully constructs the surrogate model, the surrogate model exposes the target model to numerous white-box attacks, not only limited to the ExpMI threat.

Original: We argue that adefense against ExpMI-3 should focus on mitigating the ME threat.
Revised: We argue that a defense against ExpMI-3 should focus on mitigating the ME threat.

Original: We implemented a simple MEdefense, reverse sigmoid perturbation (RSP)
Revised: We implemented a simple ME defense, reverse sigmoid perturbation (RSP)

Original: [12], and evaluted its performance in terms of degradingthe inversion peformance of ExpMI-3; the SSIM score of ExpMI-3 attack against RSP-equipped
Revised: [15], and evaluted its performance in terms of degrading the inversion peformance of ExpMI-3; the SSIM score of ExpMI-3 attack against RSP-equipped

Original: CelebA was 0.4928, which is comparable to the SSIM of ExpMI attack against GNIME-equipped
Revised: CelebA was 0.4928, which is comparable to the SSIM of ExpMI attack against GNIME-equipped

Original: CelebA-LRP, 0.4960.
Revised: CelebA-LRP, 0.4960.

Original: Refer to the further experimental results in the supplementary material.
Revised: Refer to the further experimental results in the supplementary material.

Original: Black-box adversary.
Revised: Black-box Adversary.

Original: Previous MI attacks [4, 26] have been proposed assuming the white-boxadversary who can accessto the gradients of a target model.
Revised: Previous MI attacks [5, 31] have been proposed assuming a white-box adversary who can access the gradients of a target model.

Original: By contrast, we assume the black-boxadversary conducting ExpMI attacks, following Zhao et al.
Revised: By contrast, we assume the black-box adversary conducting ExpMI attacks, following Zhao et al.

Original: Therefore, GNIME is designed tomitigate the ExpMI threat in a black-box manner in which a target model returns its explanation andoutput upon a given input.
Revised: Therefore, GNIME is designed to mitigate the ExpMI threat in a black-box manner in which a target model returns its explanation and output upon a given input.

Original: Considering that many MLaaS services operate in a black-box setting,we emphasize that addressing the ExpMI threat in this setting is still important.
Revised: Considering that many Machine-Learning-as-a-Service (MLaaS) services operate in a black-box setting, we emphasize that addressing the ExpMI threat in this setting is still important.

Original: To our knowledge, GNIME is the first defense approach to effectively mitigate the ExpMI threat.
Revised: To our knowledge, GNIME is the first defense approach to effectively mitigate the ExpMI threat.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose the first defense framework against MI attacks using model explanations. 
Revised: In this paper, we propose the first defense framework against MI attacks using model explanations. In

Original: In GNIME, we put together the noise generator and inversion networks so that they compete witheach other; the noise generator network learns to inject imperceptible noise into model explanationsthat undermine model inversion, and the inversion network learns to excel at reconstructing originalinputs from model explanations.
Revised: GNIME, we put together the noise generator and inversion networks so that they compete with each other; the noise generator network learns to inject imperceptible noise into model explanations that undermine model inversion, and the inversion network learns to excel at reconstructing the original inputs from model explanations.

Original: GNIME then leverages the trained noise generator network for everymodel explanation upon its release.
Revised: GNIME then leverages the trained noise generator network for every model explanation upon its release.

Original: Our evaluations across diverse datasets and explanation typesdemonstrate that GNIME is effective in mitigating the privacy threat that MI attacks impose whilepreserving the original functionality of model explanations.Checklist
Revised: Our evaluations across diverse datasets and explanation types demonstrate that GNIME is effective in mitigating the privacy threat that MI attacks impose while preserving the original functionality of model explanations.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?

Original: [Yes] , we summarized our contributions in the abstract and

Original: introduction.
Revised: Acknowledgements.

Original: (b) Did you describe the limitations of your work?

Original: [Yes] , we specified the limitations in

Original: Section 6. (c) Did you discuss any potential negative societal impacts of your work? [N/A] , we designed and implemented a defense framework to mitigate MI attacks, which pose no
Revised: This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-00153, Penetration

Original: threats or negative impacts to the public.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes] , we follow the ethics guideline.

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes] , we will submit

Original: our artifact in the supplemental materials and a PDF file that describes additional

Original: experimental results.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were

Original: chosen)?

Original: [Yes] , see Section 5.1.

Original: More details will be provided in the supplementary

Original: material. (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: Security Testing of ML Model Vulnerabilities and Defense).

Original: [Yes] , we conducted each MI attack five times and reported

Original: average metrics with standard deviations.

Original: Please reference Tables 1 and 2.

Original: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., type

Original: of GPUs, internal cluster, or cloud provider)?

Original: [Yes] , We described the computation

Original: resources in Section 5.1.

Original: We also describe further specifications in the supplementary

Original: material.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] , we used (1)

Original: CelebA [13], (2) MNIST [11], (3) CIFAR-10 [10], and (4) FaceScrub [16] to evaluateGNIME.

Original: We cited all the datasets in the paper.

Original: (b) Did you mention the license of the assets?

Original: [Yes] , we mentioned that all datasets are

Original: available for research in Section 5.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] ,

Original: we do not relase any new dataset.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using

Original: curating contains personally identifiable

Original: information or offensive content?

Original: N/A

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]