Original: Amplifying Membership Exposure via Data Poisoning
Revised: Amplifying Membership Exposure via Data Poisoning

Original: Abstract
Revised: Abstract

Original: As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks.
Revised: As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks.

Original: Such attacks typically lead to test-time accuracy degradation or controlled misprediction.
Revised: Such attacks typically lead to test-time accuracy degradation or controlled misprediction.

Original: In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples.
Revised: In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples.

Original: To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class.
Revised: To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class.

Original: We ﬁrst propose a generic dirty-label attack for supervised classiﬁcation algorithms.
Revised: We ﬁrst propose a generic dirty-label attack for supervised classiﬁcation algorithms.

Original: We then propose an optimization-based clean-label attack in the transfer learning scenario, whereby the poisoning samples are correctly labeled and look “natural” to evade human moderation.
Revised: We then propose an optimization-based clean-label attack in the transfer learning scenario, whereby the poisoning samples are correctly labeled and look “natural” to evade human moderation.

Original: We extensively evaluate our attacks on computer vision benchmarks.
Revised: We extensively evaluate our attacks on computer vision benchmarks.

Original: Our results show that the proposed attacks can substantially increase the membership inference precision with minimum overall test-time model performance degradation.
Revised: Our results show that the proposed attacks can substantially increase the membership inference precision with minimum overall test-time model performance degradation.

Original: To mitigate the potential negative impacts of our attacks, we also investigate feasible countermeasures.
Revised: To mitigate the potential negative impacts of our attacks, we also investigate feasible countermeasures.

Original:  Introduction
Revised:  Introduction

Original: Training data are the most critical ingredients of machine learning, which are sometimes regarded as a new type of fuel in the era of artiﬁcial intelligence [30].
Revised: Training data are the most critical ingredients of machine learning, which are sometimes regarded as a new type of fuel in the era of artiﬁcial intelligence [30].

Original: Over the last decade, training data collection and preservation have received growing concerns [29, 10] among which two focal agendas arise.
Revised: Over the last decade, training data collection and preservation have received growing concerns [29, 10] among which two focal agendas arise.

Original: One is data corruption caused by data poisoning attacks, which pose serious threats to training data integrity .
Revised: One is data corruption caused by data poisoning attacks, which pose serious threats to training data integrity .

Original: Data poisoning attacks exploit the common practice to (usually automatically) collect training data in the wild, e.g., from the Internet.
Revised: Data poisoning attacks exploit the common practice to (usually automatically) collect training data in the wild, e.g., from the Internet.

Original: This practice opens doors for attackers to inject malicious data at the training time to manipulate model behaviors at the test time.
Revised: This practice opens doors for attackers to inject malicious data at the training time to manipulate model behaviors at the test time.

Original: Consequently, models trained with poisoned training data suffer from either accuracy degradation [7, 19], targeted misclassiﬁcation [34, 55] or backdoor implantation [32, 50].
Revised: Consequently, models trained with poisoned training data suffer from either accuracy degradation [7, 19], targeted misclassiﬁcation [34, 55] or backdoor implantation [32, 50].

Original: The other is data leakage caused by privacy inference attacks, which mainly violate training data conﬁdentiality .
Revised: The other is data leakage caused by privacy inference attacks, which mainly violate training data conﬁdentiality .

Original: Ideally, a machine learning model learns generalizable knowledge of the training data, rather than details of speciﬁc data points.
Revised: Ideally, a machine learning model learns generalizable knowledge of the training data, rather than details of speciﬁc data points.

Original: However, previous studies show that conventional learning algorithms can unintentionally remember sensitive information.
Revised: However, previous studies show that conventional learning algorithms can unintentionally remember sensitive information.

Original: Such information can later be inferred by various privacy inference attacks, such as membership inference attacks [38, 33].
Revised: Such information can later be inferred by various privacy inference attacks, such as membership inference attacks [38, 33].

Original: Essentially, data poisoning is a training time attack, while privacy inference is performed at the testing time.
Revised: Essentially, data poisoning is a training time attack, while privacy inference is performed at the testing time.

Original: Despite intensive research efforts, most existing works study them separately.
Revised: Despite intensive research efforts, most existing works study them separately.

Original: It remains unclear whether these two attacks can be integrated to cause more severe damage to machine learning models.
Revised: It remains unclear whether these two attacks can be integrated to cause more severe damage to machine learning models.

Original: In this paper, we aim to advance the research frontier on the connection between data integrity and conﬁdentiality.
Revised: In this paper, we aim to advance the research frontier on the connection between data integrity and conﬁdentiality.

Original: To this end, we propose a set of poisoning attacks to increase the precision of membership inference attacks, which is widely adopted as a standard tool to measure privacy leakage in statistical analysis [40, 15].
Revised: To this end, we propose a set of poisoning attacks to increase the precision of membership inference attacks, which is widely adopted as a standard tool to measure privacy leakage in statistical analysis [40, 15].

Original: In particular, we ﬁrst show a simple but effective dirty-label poisoning attack that is generic for supervised classiﬁcation applications.
Revised: In particular, we ﬁrst show a simple but effective dirty-label poisoning attack that is generic for supervised classiﬁcation applications.

Original: We then demonstrate a clean-label poisoning attack applicable to the transfer learning scenario.
Revised: We then demonstrate a clean-label poisoning attack applicable to the transfer learning scenario.

Original: It has two signiﬁcant advantages: (1) no requirements on the labeling process and (2) natural appearances to human moderation.
Revised: It has two signiﬁcant advantages: (1) no requirements on the labeling process and (2) natural appearances to human moderation.

Original: Consequently, our proposed clean-label poisoning attack challenges the case where automatically crawling data from the Internet has become a common practice.
Revised: Consequently, our proposed clean-label poisoning attack challenges the case where automatically crawling data from the Internet has become a common practice.

Original: Our contributions are as follows: (1) We reveal an underexplored data poisoning attack threatening the training data privacy.
Revised: Our contributions are as follows: (1) We reveal an underexplored data poisoning attack threatening the training data privacy.

Original: In particular, we demonstrate how to amplify membership inference exposure of a speciﬁc class with only slight impacts on the model performance.
Revised: In particular, we demonstrate how to amplify membership inference exposure of a speciﬁc class with only slight impacts on the model performance.

Original: (2) In generic supervised learning settings, we introduce a naïve dirty-label poisoning attack by modifying the labels of the poisoning samples.
Revised: (2) In generic supervised learning settings, we introduce a naïve dirty-label poisoning attack by modifying the labels of the poisoning samples.

Original: In the transfer learning setting, we propose an optimization-based clean-label poisoning attack, wherein for each poisoning sample, we make imperceptible modiﬁcations without changing its label.
Revised: In the transfer learning setting, we propose an optimization-based clean-label poisoning attack, wherein for each poisoning sample, we make imperceptible modiﬁcations without changing its label.

Original: It presents a more practical attack example since it does not require control over the labeling process.
Revised: It presents a more practical attack example since it does not require control over the labeling process.

Original: It is also a more stealthy attack as the contents of poisoning instances “look similar” to the natural ones.
Revised: It is also a more stealthy attack as the contents of poisoning instances “look similar” to the natural ones.

Original: (4) We conduct extensive empirical studies to investigate the impacts of various factors, including datasets, architectures, poisoning budgets, and learning setups.
Revised: (4) We conduct extensive empirical studies to investigate the impacts of various factors, including datasets, architectures, poisoning budgets, and learning setups.

Original: Our results show that our proposed poisoning attacks are able to increase the membership exposure obviously with just limited inﬂuences on the victim model’s performance on testing samples.
Revised: Our results show that our proposed poisoning attacks are able to increase the membership exposure obviously with just limited inﬂuences on the victim model’s performance on testing samples.

Original: At the end, we also consider several potential countermeasures and investigate their effectiveness.
Revised: At the end, we also consider several potential countermeasures and investigate their effectiveness.

Original:  Background
Revised:  Background

Original: In this work, we focus on deep supervised classiﬁcation in the computer vision ﬁeld.
Revised: In this work, we focus on deep supervised classiﬁcation in the computer vision ﬁeld.

Original: That is, given a training dataset D train composed of input-label pairs ( x, y ) , a deep classiﬁer f ( x ; ✓ ) with model parameters ✓ , and a learning objective L ( f ( x ; ✓ ) , y ) , the learning process aims to ﬁnd a set of model parameters ✓ ⇤ to minimize the learning objective on D train , i.e., ✓ ⇤ = arg min ✓
Revised: That is, given a training dataset D train composed of input-label pairs ( x, y ) , a deep classiﬁer f ( x ; ✓ ) with model parameters ✓ , and a learning objective L ( f ( x ; ✓ ) , y ) , the learning process aims to ﬁnd a set of model parameters ✓ ⇤ to minimize the learning objective on D train , i.e., ✓ ⇤ = arg min ✓

Original: [Equation]
Revised: [Equation]

Original: 2.1 Data Poisoning
Revised: 2.1 Data Poisoning

Original: In a poisoning attack, an attacker crafts and injects a set of malicious training samples D poison (i.e., poisoning dataset ) into the benign dataset D clean (i.e., clean dataset ).
Revised: In a poisoning attack, an attacker crafts and injects a set of malicious training samples D poison (i.e., poisoning dataset ) into the benign dataset D clean (i.e., clean dataset ).

Original: In the training stage, the model holder executes the machine learning algorithm on the full poisoned dataset D train =
Revised: In the training stage, the model holder executes the machine learning algorithm on the full poisoned dataset D train =

Original: D clean [ D poison to obtain the trained model f
Revised: D clean [ D poison to obtain the trained model f

Original: ( x ; ✓ ⇤ ) .
Revised: ( x ; ✓ ⇤ ) .

Original: In the inference stage, f ( x ; ✓ ⇤ ) tends to show unexpected behaviors on targeted inputs ( x, y ) 2 D target .
Revised: In the inference stage, f ( x ; ✓ ⇤ ) tends to show unexpected behaviors on targeted inputs ( x, y ) 2 D target .

Original: Most existing literature poses the poisoning attack as a bi-level optimization problem [8, 16]:
Revised: Most existing literature poses the poisoning attack as a bi-level optimization problem [8, 16]:

Original: [Equation]
Revised: [Equation]

Original: where A is the adversarial objective of the poisoning attack.
Revised: where A is the adversarial objective of the poisoning attack.

Original: Typically, there are three adversarial objectives: (1) in the accuracy degradation case, D target contains all testing samples and
Revised: Typically, there are three adversarial objectives: (1) in the accuracy degradation case, D target contains all testing samples and

Original: A =
Revised: A =

Original:  L ( f ( x ; ✓ ⇤ ) , y ) ; (2) in the targeted misclassiﬁcation case, D target contains samples expected to be misclassiﬁed into the target class t and A =
Revised:  L ( f ( x ; ✓ ⇤ ) , y ) ; (2) in the targeted misclassiﬁcation case, D target contains samples expected to be misclassiﬁed into the target class t and A =

Original: L ( f ( x ; ✓ ⇤ ) , t ) ; (3) in the backdoor implantation case,
Revised: L ( f ( x ; ✓ ⇤ ) , t ) ; (3) in the backdoor implantation case,

Original: D target involves samples with a trigger  , where inputs embedded with the trigger are to be classiﬁed into the target class t and A =
Revised: D target involves samples with a trigger  , where inputs embedded with the trigger are to be classiﬁed into the target class t and A =

Original: L ( f ( x   ; ✓ ⇤ ) , t ) .
Revised: L ( f ( x   ; ✓ ⇤ ) , t ) .

Original: Based on the attacker’s capability, poisoning attacks can be further grouped into two categories:
Revised: Based on the attacker’s capability, poisoning attacks can be further grouped into two categories:

Original: • Dirty-label poisoning.
Revised: • Dirty-label poisoning.

Original: Most classical poisoning attacks require the control of the labeling process, where the attacker is allowed to modify the labels in D poison [19, 48, 42].
Revised: Most classical poisoning attacks require the control of the labeling process, where the attacker is allowed to modify the labels in D poison [19, 48, 42].

Original: Such an attack is called dirty-label poisoning .
Revised: Such an attack is called dirty-label poisoning .

Original: Despite promising attack performance, dirty-label poisoning becomes impractical in supervised machine learning scenarios for two main reasons.
Revised: Despite promising attack performance, dirty-label poisoning becomes impractical in supervised machine learning scenarios for two main reasons.

Original: First, in reality, it is a common practice that only unlabeled data (e.g., images and videos) are scraped by crawlers and then labeled by human moderators.
Revised: First, in reality, it is a common practice that only unlabeled data (e.g., images and videos) are scraped by crawlers and then labeled by human moderators.

Original: Second, machine learning developers usually utilize anomaly detection algorithms to ﬁlter out wrongly labeled samples.
Revised: Second, machine learning developers usually utilize anomaly detection algorithms to ﬁlter out wrongly labeled samples.

Original: Therefore, there exists a minimal chance that the data samples with modiﬁed labels are preserved in the dataset.
Revised: Therefore, there exists a minimal chance that the data samples with modiﬁed labels are preserved in the dataset.

Original: • Clean-label poisoning.
Revised: • Clean-label poisoning.

Original: To overcome the shortcomings of dirty-label poisoning, recent studies propose clean-label poisoning attacks [34, 53, 55].
Revised: To overcome the shortcomings of dirty-label poisoning, recent studies propose clean-label poisoning attacks [34, 53, 55].

Original: A clean-label poisoning attack
Revised: A clean-label poisoning attack

Original: satisﬁes two properties.
Revised: satisﬁes two properties.

Original: First, the labels of poisoning samples remain unchanged in the poisoning process.
Revised: First, the labels of poisoning samples remain unchanged in the poisoning process.

Original: Second, to be inconspicuous, each poisoning sample x p visually resembles a natural sample x n , which is always constrained by a L p -norm distance
Revised: Second, to be inconspicuous, each poisoning sample x p visually resembles a natural sample x n , which is always constrained by a L p -norm distance

Original: k x p  x n k p < ✏ .
Revised: k x p  x n k p < ✏ .

Original: Our setup.
Revised: Our setup.

Original: In this work, we investigate data poisoning under a newly identiﬁed and underexplored adversarial objective: privacy leakage ampliﬁcation .
Revised: In this work, we investigate data poisoning under a newly identiﬁed and underexplored adversarial objective: privacy leakage ampliﬁcation .

Original: In particular, we focus on one of the most representative privacy inference attacks: membership inference attack [38, 33].
Revised: In particular, we focus on one of the most representative privacy inference attacks: membership inference attack [38, 33].

Original: We start from the dirty-label poisoning setup, which establishes a baseline and cornerstone for more advanced attacks.
Revised: We start from the dirty-label poisoning setup, which establishes a baseline and cornerstone for more advanced attacks.

Original: Then, we investigate attack methods in the clean-label poisoning setup to improve practicability.
Revised: Then, we investigate attack methods in the clean-label poisoning setup to improve practicability.

Original: Note that there are concurrent works that propose to use poisoning attacks to cause privacy leakage [26, 43].
Revised: Note that there are concurrent works that propose to use poisoning attacks to cause privacy leakage [26, 43].

Original: 2.2 Membership Inference Attack
Revised: 2.2 Membership Inference Attack

Original: In a membership inference attack, an attacker aims to infer whether a speciﬁc sample ( x, y ) belongs to the training dataset D train at the test time [38, 33, 24].
Revised: In a membership inference attack, an attacker aims to infer whether a speciﬁc sample ( x, y ) belongs to the training dataset D train at the test time [38, 33, 24].

Original: Unintended membership exposure causes catastrophic privacy loss for individuals.
Revised: Unintended membership exposure causes catastrophic privacy loss for individuals.

Original: For example, in the real world, a data sample x can be a clinical record or an individual.
Revised: For example, in the real world, a data sample x can be a clinical record or an individual.

Original: Membership inference attacks enable the attackers to infer whether this clinical record or individual has been used to train a model associated with a certain disease.
Revised: Membership inference attacks enable the attackers to infer whether this clinical record or individual has been used to train a model associated with a certain disease.

Original: As such, these attacks are widely adopted as basic metrics to quantify privacy exposure in statistical data analysis algorithms [13, 23].
Revised: As such, these attacks are widely adopted as basic metrics to quantify privacy exposure in statistical data analysis algorithms [13, 23].

Original: Henceforth, we use membership inference attacks to demonstrate how to exploit data poisoning to amplify privacy leakage in this paper.
Revised: Henceforth, we use membership inference attacks to demonstrate how to exploit data poisoning to amplify privacy leakage in this paper.

Original: Based on the attacker’s capability, membership inference attacks can be grouped into two categories:
Revised: Based on the attacker’s capability, membership inference attacks can be grouped into two categories:

Original: • Black-box membership inference.
Revised: • Black-box membership inference.

Original: In this case, the attacker distinguishes members and non-members only using model outputs [38, 37, 17].
Revised: In this case, the attacker distinguishes members and non-members only using model outputs [38, 37, 17].

Original: This case is generic to most machine learning contexts.
Revised: This case is generic to most machine learning contexts.

Original: There are two attack strategies in general.
Revised: There are two attack strategies in general.

Original: The ﬁrst strategy is modelbased [38], where the attacker builds multiple shadow models to mimic the victim model, then utilizes them to construct a dataset with member/non-member labels, and ﬁnally trains a binary classiﬁer to predict member/non-member.
Revised: The ﬁrst strategy is modelbased [38], where the attacker builds multiple shadow models to mimic the victim model, then utilizes them to construct a dataset with member/non-member labels, and ﬁnally trains a binary classiﬁer to predict member/non-member.

Original: The second strategy is metricbased [33, 39], where the attacker compares a designed metric M mem , such as correctness or entropy, with a predeﬁned threshold ⌧ to infer if a sample belongs to the training dataset.
Revised: The second strategy is metricbased [33, 39], where the attacker compares a designed metric M mem , such as correctness or entropy, with a predeﬁned threshold ⌧ to infer if a sample belongs to the training dataset.

Original: • White-box membership inference.
Revised: • White-box membership inference.

Original: In this case, model parameters ✓ ⇤ or even intermediate training information such as gradients @
Revised: In this case, model parameters ✓ ⇤ or even intermediate training information such as gradients @

Original: L @✓ are observable by the attacker [27, 31, 22].
Revised: L @✓ are observable by the attacker [27, 31, 22].

Original: Such capability provides additional information supporting inference attacks, which is usually achievable in collaborative learning settings.
Revised: Such capability provides additional information supporting inference attacks, which is usually achievable in collaborative learning settings.

Original: Most white-box attacks are model-based, as the attacker needs to access the internals of deep models to extract model-speciﬁc features.
Revised: Most white-box attacks are model-based, as the attacker needs to access the internals of deep models to extract model-speciﬁc features.

Original: Our setup.
Revised: Our setup.

Original: Our evaluation is carried out in the metric-based black-box membership inference setting.
Revised: Our evaluation is carried out in the metric-based black-box membership inference setting.

Original: We adopt the following metric proposed by Song et al.
Revised: We adopt the following metric proposed by Song et al.

Original: [40] to measure the membership exposure 2 :
Revised: [40] to measure the membership exposure 2 :

Original: [Equation]
Revised: [Equation]

Original: where f ( x ) j refers to the conﬁdence value of label j .
Revised: where f ( x ) j refers to the conﬁdence value of label j .

Original: Equation 2 simultaneously considers the correctness and entropy metric.
Revised: Equation 2 simultaneously considers the correctness and entropy metric.

Original: In the training stage, the learning algorithm continuously ﬁts the training samples, by decreasing the entropy loss (i.e., the learning objective) and increasing the conﬁdence score of the correct label.
Revised: In the training stage, the learning algorithm continuously ﬁts the training samples, by decreasing the entropy loss (i.e., the learning objective) and increasing the conﬁdence score of the correct label.

Original: As a result, a member is likely to produce a higher M mem than a non-member.
Revised: As a result, a member is likely to produce a higher M mem than a non-member.

Original: To gain a holistic view of the membership exposure, we calculate the true positive rate (TPR) and false positive rate (FPR) of the membership inference attack by varying the threshold ⌧ , and plot the ROC curve.
Revised: To gain a holistic view of the membership exposure, we calculate the true positive rate (TPR) and false positive rate (FPR) of the membership inference attack by varying the threshold ⌧ , and plot the ROC curve.

Original: Then We adopt the AUC (area under the ROC curve) score to measure membership exposure.
Revised: Then We adopt the AUC (area under the ROC curve) score to measure membership exposure.

Original: In general, a higher AUC score means a higher risk of membership exposure, as we can ﬁnd a threshold ⌧ with high TPR and low FPR.
Revised: In general, a higher AUC score means a higher risk of membership exposure, as we can ﬁnd a threshold ⌧ with high TPR and low FPR.

Original: It is notable that the AUC score is an average-case metric, which hardly effectively reﬂects the worst-case privacy [9].
Revised: It is notable that the AUC score is an average-case metric, which hardly effectively reﬂects the worst-case privacy [9].

Original: Therefore, we also report the TPR when the FPR is low (1% in our case), which works as a compensation for the AUC metric to indicate the worst-case privacy.
Revised: Therefore, we also report the TPR when the FPR is low (1% in our case), which works as a compensation for the AUC metric to indicate the worst-case privacy.

Original: We also evaluate our poisoning attacks with a stronger membership inference attack [9] on the CIFAR-10 dataset.
Revised: We also evaluate our poisoning attacks with a stronger membership inference attack [9] on the CIFAR-10 dataset.

Original: The experimental results show that our poisoning attacks can also help improve the membership
Revised: The experimental results show that our poisoning attacks can also help improve the membership

Original: 2.3 Threat Model
Revised: 2.3 Threat Model

Original: Before diving into our attack design, we ﬁrst elaborate on the threat model considered in this paper.
Revised: Before diving into our attack design, we ﬁrst elaborate on the threat model considered in this paper.

Original: Attack Goals.
Revised: Attack Goals.

Original: The ﬁrst goal of the attacker is increasing the chance of leaking the membership of training samples within a targeted class .
Revised: The ﬁrst goal of the attacker is increasing the chance of leaking the membership of training samples within a targeted class .

Original: At the same time, the attacker attempts to make the poisoning attack as stealthy as possible.
Revised: At the same time, the attacker attempts to make the poisoning attack as stealthy as possible.

Original: As such, the second goal of the attacker is generating poisoning samples that have limited impacts on the model performance for untargeted classes and are indistinguishable to natural samples .
Revised: As such, the second goal of the attacker is generating poisoning samples that have limited impacts on the model performance for untargeted classes and are indistinguishable to natural samples .

Original: Attacker Capabilities.
Revised: Attacker Capabilities.

Original: We assume the attacker has the basic capabilities set up by existing data poisoning and membership inference attack games.
Revised: We assume the attacker has the basic capabilities set up by existing data poisoning and membership inference attack games.

Original: The attacker can craft poisoning samples and inject them into the victim’s clean dataset D clean .
Revised: The attacker can craft poisoning samples and inject them into the victim’s clean dataset D clean .

Original: However, there exists a poisoning budget b poison to limit the amount of training samples ( |D poison |  b poison ⌧ |D clean | ).
Revised: However, there exists a poisoning budget b poison to limit the amount of training samples ( |D poison |  b poison ⌧ |D clean | ).

Original: We assume the attacker owns a shadow dataset
Revised: We assume the attacker owns a shadow dataset

Original: D shadow to craft poisoning samples, which contains natural samples from the same distribution with D clean .
Revised: D shadow to craft poisoning samples, which contains natural samples from the same distribution with D clean .

Original: Moreover, the attacker cannot modify poisoning data labels in the clean-label poisoning setting.
Revised: Moreover, the attacker cannot modify poisoning data labels in the clean-label poisoning setting.

Original: But we assume the attacker knows the feature extractor (i.e., an encoder) used by the victim model.
Revised: But we assume the attacker knows the feature extractor (i.e., an encoder) used by the victim model.

Original: This assumption is practical since developing models with public pretrained feature extractors is a common practice in existing clean-label poisoning literature [34, 53].
Revised: This assumption is practical since developing models with public pretrained feature extractors is a common practice in existing clean-label poisoning literature [34, 53].

Original: Remark.
Revised: Remark.

Original: In our setup, the clean dataset D clean is unknown by the attacker, which is different from most existing data poisoning games [7, 19, 26].
Revised: In our setup, the clean dataset D clean is unknown by the attacker, which is different from most existing data poisoning games [7, 19, 26].

Original:  Attack Methodology
Revised:  Attack Methodology

Original: Although Equation 1 establishes a generic framework for poisoning attack, it is unsuitable for our case.
Revised: Although Equation 1 establishes a generic framework for poisoning attack, it is unsuitable for our case.

Original: First, classic solutions for the optimization problem posed by Equation 1 require differentiation to the inner loss minimizer, which is intractable to models with high complexity.
Revised: First, classic solutions for the optimization problem posed by Equation 1 require differentiation to the inner loss minimizer, which is intractable to models with high complexity.

Original: Second, although meta-learning algorithms have been proposed to solve the bi-level optimization problem in the deep learning setting [16, 52, 54], they lead to huge computing costs, limiting the practicability of the attack.
Revised: Second, although meta-learning algorithms have been proposed to solve the bi-level optimization problem in the deep learning setting [16, 52, 54], they lead to huge computing costs, limiting the practicability of the attack.

Original: Third, in our threat model, D clean is unobservable, making Equation 1 no longer applicable to our attack.
Revised: Third, in our threat model, D clean is unobservable, making Equation 1 no longer applicable to our attack.

Original: Facing these obstacles, we explore heuristic strategies to achieve the attack goals.
Revised: Facing these obstacles, we explore heuristic strategies to achieve the attack goals.

Original: 3.1 Dirty-label Poisoning Attack: A Naïve Approach
Revised: 3.1 Dirty-label Poisoning Attack: A Naïve Approach

Original: [Figure]
Revised: [Figure]

Original: We start our investigation in the dirty-label poisoning setting.
Revised: We start our investigation in the dirty-label poisoning setting.

Original: This case has more limitations in the real world as discussed in Section 2.1.
Revised: This case has more limitations in the real world as discussed in Section 2.1.

Original: But it relaxes the attack constraints and helps to verify the feasibility of our attack strategy.
Revised: But it relaxes the attack constraints and helps to verify the feasibility of our attack strategy.

Original: The key to amplifying membership exposure is to cause overﬁtting in the targeted class.
Revised: The key to amplifying membership exposure is to cause overﬁtting in the targeted class.

Original: We propose the following attack strategy:
Revised: We propose the following attack strategy:

Original: Our dirty-label poisoning attack.
Revised: Our dirty-label poisoning attack.

Original: Given the label of the target group t , the shadow dataset
Revised: Given the label of the target group t , the shadow dataset

Original: D shadow , and a poisoning budget b poison , the poisoning dataset is constructed by the following steps: (1) select all samples ( x, t ) 2 D shadow with label t ; (2) for each sample ( x, t ) , randomly change the label to another class i 6 = t ; (3) preserve at most b poison samples ( x, i ) to compose D poison .
Revised: D shadow , and a poisoning budget b poison , the poisoning dataset is constructed by the following steps: (1) select all samples ( x, t ) 2 D shadow with label t ; (2) for each sample ( x, t ) , randomly change the label to another class i 6 = t ; (3) preserve at most b poison samples ( x, i ) to compose D poison .

Original: Intuitively, only learning general concepts/features is insufﬁcient to discriminate the clean samples of class t from poisoning samples, since in fact, they share similar features.
Revised: Intuitively, only learning general concepts/features is insufﬁcient to discriminate the clean samples of class t from poisoning samples, since in fact, they share similar features.

Original: To minimize the training loss, the victim model has to learn more speciﬁc features of each sample.
Revised: To minimize the training loss, the victim model has to learn more speciﬁc features of each sample.

Original: As conceptually illustrated by Figure 1, the generalization performance on class t tend to degrade.
Revised: As conceptually illustrated by Figure 1, the generalization performance on class t tend to degrade.

Original: Figure 2 gives an example to show how our poisoning attack improves the membership exposure of the airplane category for an inference accuracy by a stronger membership inference attacker.
Revised: Figure 2 gives an example to show how our poisoning attack improves the membership exposure of the airplane category for an inference accuracy by a stronger membership inference attacker.

Original: We recommend interested readers ﬁnd more results in the appendix.
Revised: We recommend interested readers ﬁnd more results in the appendix.

Original: In the main body of the paper, our experiments only assume the weak attack used by [40], as it has lower computation costs and is more practical.
Revised: In the main body of the paper, our experiments only assume the weak attack used by [40], as it has lower computation costs and is more practical.

Original: InceptionV3-based CIFAR-10 classiﬁer.
Revised: InceptionV3-based CIFAR-10 classiﬁer.

Original: In this example, we assume the numbers of members and non-members are both 10,000 and b poison =1,000.
Revised: In this example, we assume the numbers of members and non-members are both 10,000 and b poison =1,000.

Original: [Figure]
Revised: [Figure]

Original: Remark.
Revised: Remark.

Original: We do not pose extra assumptions to the training process in this case.
Revised: We do not pose extra assumptions to the training process in this case.

Original: Our dirty-label poisoning attack is generic to all supervised learning scenarios, including end-to-end learning and transfer learning.
Revised: Our dirty-label poisoning attack is generic to all supervised learning scenarios, including end-to-end learning and transfer learning.

Original: Similar to the concurrent work [43], we also use the classical label ﬂipping strategy [7] to carry out the dirty-label attack.
Revised: Similar to the concurrent work [43], we also use the classical label ﬂipping strategy [7] to carry out the dirty-label attack.

Original: 3.2 Clean-label Poisoning Attack: A Stealthy Approach
Revised: 3.2 Clean-label Poisoning Attack: A Stealthy Approach

Original: In this part, we aim to improve our poisoning attack to amplify membership exposure while attaining stealthiness.
Revised: In this part, we aim to improve our poisoning attack to amplify membership exposure while attaining stealthiness.

Original: Following the basic setup in [34], we consider the transfer learning setting.
Revised: Following the basic setup in [34], we consider the transfer learning setting.

Original: In transfer learning, we assume the victim model is composed of two parts: a pretrained feature extractor g ( · ) to extract high-level features from the input x and a newly-trained classiﬁer c ( · ) to predict labels.
Revised: In transfer learning, we assume the victim model is composed of two parts: a pretrained feature extractor g ( · ) to extract high-level features from the input x and a newly-trained classiﬁer c ( · ) to predict labels.

Original: The key to our clean-label poisoning is to exploit one vulnerability of deep feature extractors: a slight change in the input space may cause a signiﬁcant change in the feature space.
Revised: The key to our clean-label poisoning is to exploit one vulnerability of deep feature extractors: a slight change in the input space may cause a signiﬁcant change in the feature space.

Original: We propose the following attack strategy.
Revised: We propose the following attack strategy.

Original: Our clean-label poisoning attack.
Revised: Our clean-label poisoning attack.

Original: Given the label of the target class t , the shadow dataset D shadow , a poisoning budget b poison , and the adopted feature extractor g , the poisoning dataset is constructed by the following steps: (1) from D shadow , select a base sample ( x base , t ) with label t and a sample ( x, y ) where y 6 = t ; (2) ﬁnd a x ⇤ that is close to x in the input space ( k x ⇤  x k p  ✏ ) and close to x base in the feature space ( g ( x ⇤ ) ⇡ g ( x base ) ), and insert ( x ⇤ , y ) to D poison ; (3) repeat step (1) and (2) until |D poison | =
Revised: Given the label of the target class t , the shadow dataset D shadow , a poisoning budget b poison , and the adopted feature extractor g , the poisoning dataset is constructed by the following steps: (1) from D shadow , select a base sample ( x base , t ) with label t and a sample ( x, y ) where y 6 = t ; (2) ﬁnd a x ⇤ that is close to x in the input space ( k x ⇤  x k p  ✏ ) and close to x base in the feature space ( g ( x ⇤ ) ⇡ g ( x base ) ), and insert ( x ⇤ , y ) to D poison ; (3) repeat step (1) and (2) until |D poison | =

Original: b poison .
Revised: b poison .

Original: The intuition of our clean-label poisoning attack is straightforward: we actually mount a dirty-label poisoning attack in the feature space.
Revised: The intuition of our clean-label poisoning attack is straightforward: we actually mount a dirty-label poisoning attack in the feature space.

Original: That is, the poisoning sample ( x ⇤ , y ) is close to ( x base , t ) , a clean sample from class t , in the feature space but labeled as y .
Revised: That is, the poisoning sample ( x ⇤ , y ) is close to ( x base , t ) , a clean sample from class t , in the feature space but labeled as y .

Original: We formalize step (2) as a constrained optimization problem:
Revised: We formalize step (2) as a constrained optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where X refers to the input space.
Revised: where X refers to the input space.

Original: For instance, for a normalized RGB image input, a valid pixel value in one color channel should be within [0 , 1] .
Revised: For instance, for a normalized RGB image input, a valid pixel value in one color channel should be within [0 , 1] .

Original: For ease of optimization, we adopt L 2 -norm to measure the distance in the feature space.
Revised: For ease of optimization, we adopt L 2 -norm to measure the distance in the feature space.

Original: Following the convention of adversarial attack work, we adopt L 1 to measure the distance in the input space.
Revised: Following the convention of adversarial attack work, we adopt L 1 to measure the distance in the input space.

Original: In our case, the constraint can be expressed as x 0 2 [ x min , x max ] , where x min = max(0 , x  ✏ ) , x max = min(1 , x + ✏ ) .
Revised: In our case, the constraint can be expressed as x 0 2 [ x min , x max ] , where x min = max(0 , x  ✏ ) , x max = min(1 , x + ✏ ) .

Original: Optimization.
Revised: Optimization.

Original: We adopt the variable substitution method introduced by [11] to solve Equation 3.
Revised: We adopt the variable substitution method introduced by [11] to solve Equation 3.

Original: We introduce a new variable w and let
Revised: We introduce a new variable w and let

Original: [Equation]
Revised: [Equation]

Original: Based on the fact that tanh( w ) 2 [  1 , 1] , we can prove that the constraint x 0 2 [ x min , x max ] holds in Equation 4.
Revised: Based on the fact that tanh( w ) 2 [  1 , 1] , we can prove that the constraint x 0 2 [ x min , x max ] holds in Equation 4.

Original: Plugging Equation 4 into Equation 3, we can obtain
Revised: Plugging Equation 4 into Equation 3, we can obtain

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: 5) which poses an unconstrained optimization problem that can be simply solved by mainstream gradient optimization methods.
Revised: 5) which poses an unconstrained optimization problem that can be simply solved by mainstream gradient optimization methods.

Original: In our implementation, we ﬁrst initialize variable w with x base by Equationand iterate the optimization process 1,000 times, using an Adam optimizer with the learning rate 0.01.
Revised: In our implementation, we ﬁrst initialize variable w with x base by Equationand iterate the optimization process 1,000 times, using an Adam optimizer with the learning rate 0.01.

Original: Figure 3 illustrates some examples generated by our proposed two poisoning attacks.
Revised: Figure 3 illustrates some examples generated by our proposed two poisoning attacks.

Original: We can see that the poisoning samples produced by our clean-label poisoning method look natural to human eyes, which are hard to detect by manual moderation.
Revised: We can see that the poisoning samples produced by our clean-label poisoning method look natural to human eyes, which are hard to detect by manual moderation.

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: cat
Revised: cat

Original: [Equation]
Revised: [Equation]

Original: deer ship
Revised: deer ship

Original:  Evaluation 4.1 Experimental Setup
Revised:  Evaluation 4.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: In our experiment, we split each dataset into three portions: the clean training dataset
Revised: In our experiment, we split each dataset into three portions: the clean training dataset

Original: D clean containing the members, the test dataset D test containing the non-members, and the shadow dataset D shadow for generating poisoning samples.
Revised: D clean containing the members, the test dataset D test containing the non-members, and the shadow dataset D shadow for generating poisoning samples.

Original: We follow the same setup with [38], where |D clean | =
Revised: We follow the same setup with [38], where |D clean | =

Original: |D test | =
Revised: |D test | =

Original: |D shadow | , and each of them does not overlap with others.
Revised: |D shadow | , and each of them does not overlap with others.

Original: Additionally, we set the three datasets to be balanced for the simplicity of evaluation among each class.
Revised: Additionally, we set the three datasets to be balanced for the simplicity of evaluation among each class.

Original: We adopt ﬁve datasets for our experiments, including (1) MNIST [1] that contains 60,000 handwritten digits from 0 to 9.
Revised: We adopt ﬁve datasets for our experiments, including (1) MNIST [1] that contains 60,000 handwritten digits from 0 to 9.

Original: (2) CIFAR-10 [2] that contains 60,000 images from 10 classes.
Revised: (2) CIFAR-10 [2] that contains 60,000 images from 10 classes.

Original: (3) STL-10 [3] that contains 130, labeled images from 10 classes.
Revised: (3) STL-10 [3] that contains 130, labeled images from 10 classes.

Original: (4) CelebA [25] that contains 202,599 face images annotated by 40 attributes.
Revised: (4) CelebA [25] that contains 202,599 face images annotated by 40 attributes.

Original: In our experiment, we train a two-class classiﬁer is to predict the Charming attribute; (5) PatchCamelyon [46] that contains 327,680 images to predict presence of metastatic tissue.
Revised: In our experiment, we train a two-class classiﬁer is to predict the Charming attribute; (5) PatchCamelyon [46] that contains 327,680 images to predict presence of metastatic tissue.

Original: For
Revised: For

Original: MNIST, CIFAR-10, CelebA, and PatchCamelyon dataset, we set |D clean | =10,000, while for STL-10, we set |D clean | =4,000.
Revised: MNIST, CIFAR-10, CelebA, and PatchCamelyon dataset, we set |D clean | =10,000, while for STL-10, we set |D clean | =4,000.

Original: Each sample is transformed into a 96 ⇥ 96 ⇥ 3 RGB image.
Revised: Each sample is transformed into a 96 ⇥ 96 ⇥ 3 RGB image.

Original: Models.
Revised: Models.

Original: We use ﬁve pretrained models provided by Tensorﬂow(v2.5.2): Xception, ResNet18,
Revised: We use ﬁve pretrained models provided by Tensorﬂow(v2.5.2): Xception, ResNet18,

Original: MobileNetv2, InceptionV3, and VGG16.
Revised: MobileNetv2, InceptionV3, and VGG16.

Original: For each model, we remove the fully connected (FC) layers to build up the feature extractor g ( · ) .
Revised: For each model, we remove the fully connected (FC) layers to build up the feature extractor g ( · ) .

Original: Then, we add two FC layers on top of the feature extractor to form c ( · ) : one layer with 128 hidden units using the Tanh activation function, followed by an output layer using the Softmax activation function.
Revised: Then, we add two FC layers on top of the feature extractor to form c ( · ) : one layer with 128 hidden units using the Tanh activation function, followed by an output layer using the Softmax activation function.

Original: We ﬁx the feature extractor and train the newly added
Revised: We ﬁx the feature extractor and train the newly added

Original: FC layers with the Adam optimizer, with the learning rate of 10  3 and batch size of 100.
Revised: FC layers with the Adam optimizer, with the learning rate of 10  3 and batch size of 100.

Original: Poisoning Setup.
Revised: Poisoning Setup.

Original: For each learning task, we mount the poisoning attack against each class.
Revised: For each learning task, we mount the poisoning attack against each class.

Original: We set the poisoning budget b poison as |D clean | # classes .
Revised: We set the poisoning budget b poison as |D clean | # classes .

Original: To make our poisoning attack stealthy to the victim, we assume D poison is evenly distributed among all classes with each having approximately b poison # classes poisoned data samples.
Revised: To make our poisoning attack stealthy to the victim, we assume D poison is evenly distributed among all classes with each having approximately b poison # classes poisoned data samples.

Original: For instance, for a CIFAR-10 classiﬁer, we set b poison = 1 , 000 , where only 100 samples for each class are in D poison .
Revised: For instance, for a CIFAR-10 classiﬁer, we set b poison = 1 , 000 , where only 100 samples for each class are in D poison .

Original: Note that this assumption means that there are b poison # classes clean samples in D clean for the target class t .
Revised: Note that this assumption means that there are b poison # classes clean samples in D clean for the target class t .

Original: For the clean-label poisoning dataset, we set the perturbation constraint ✏ = 16 / 255 .
Revised: For the clean-label poisoning dataset, we set the perturbation constraint ✏ = 16 / 255 .

Original: Equipment.
Revised: Equipment.

Original: Our experiments were conducted on a deep learning server, which equipped with an
Revised: Our experiments were conducted on a deep learning server, which equipped with an

Original: Intel(R)
Revised: Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 6226R CPU @
Revised: Gold 6226R CPU @

Original: 2.90GHz, 128GB RAM, and four NVIDIA GeForce RTX 3090 GPUs with 24GB of memory.
Revised: 2.90GHz, 128GB RAM, and four NVIDIA GeForce RTX 3090 GPUs with 24GB of memory.

Original: 4.2 Results
Revised: 4.2 Results

Original: Table 1 reports the evaluation results on our proposed two poisoning attacks.
Revised: Table 1 reports the evaluation results on our proposed two poisoning attacks.

Original: In general, we can observe that our attacks obviously increase the AUC score of the membership inference against the target class t , with slight testing accuracy decay.
Revised: In general, we can observe that our attacks obviously increase the AUC score of the membership inference against the target class t , with slight testing accuracy decay.

Original: For instance, for the clean-label poisoning attack against the STL-10 Xception-based classiﬁer, the membership inference AUC is increased from 0.578 to 0.870 on average, while the testing accuracy is decreased from 0.861 to 0.837 on average.
Revised: For instance, for the clean-label poisoning attack against the STL-10 Xception-based classiﬁer, the membership inference AUC is increased from 0.578 to 0.870 on average, while the testing accuracy is decreased from 0.861 to 0.837 on average.

Original: The results show that the poisoned samples resemble the clean samples and the performance of the
Revised: The results show that the poisoned samples resemble the clean samples and the performance of the

Original: [Table]
Revised: [Table]

Original: poisoned models degrades subtly on the testing dataset.
Revised: poisoned models degrades subtly on the testing dataset.

Original: Consequently, there are chances that the clean-label poisoning attack evade the manual inspection, when the classiﬁcation performance for each class does not get carefully examined.
Revised: Consequently, there are chances that the clean-label poisoning attack evade the manual inspection, when the classiﬁcation performance for each class does not get carefully examined.

Original: Another observation is that the dirty-label poisoning attack causes more signiﬁcant membership exposure than the clean-label poisoning attack.
Revised: Another observation is that the dirty-label poisoning attack causes more signiﬁcant membership exposure than the clean-label poisoning attack.

Original: A potential explanation of this phenomenon is that our clean-label poisoning attack can be considered as an “approximate” version of the dirty-label poisoning attack in the feature space.
Revised: A potential explanation of this phenomenon is that our clean-label poisoning attack can be considered as an “approximate” version of the dirty-label poisoning attack in the feature space.

Original:  Ablation Study 5.1 Study 1: Impact of D shadow
Revised:  Ablation Study 5.1 Study 1: Impact of D shadow

Original: In our prior study we assume the attacker can obtain a shadow dataset D shadow with the same distribution of the victim’s clean training dataset D clean .
Revised: In our prior study we assume the attacker can obtain a shadow dataset D shadow with the same distribution of the victim’s clean training dataset D clean .

Original: However, the model developer may hide the training dataset information to fortify privacy and intellectual property (IP) protection.
Revised: However, the model developer may hide the training dataset information to fortify privacy and intellectual property (IP) protection.

Original: In this ablation section, we examine how the D shadow affects the attack performance.
Revised: In this ablation section, we examine how the D shadow affects the attack performance.

Original: In our experiment, we assume the attacker aims to poison the cat and aircraft categories against the
Revised: In our experiment, we assume the attacker aims to poison the cat and aircraft categories against the

Original: CIFAR-10 classiﬁers respectively.
Revised: CIFAR-10 classiﬁers respectively.

Original: For each target class t , we select 1,000 samples from t in STL-10.
Revised: For each target class t , we select 1,000 samples from t in STL-10.

Original: In this ablation study, We start with the dirty-label poisoning attack to estimate the upper bound performance of our clean-label poisoning attack.
Revised: In this ablation study, We start with the dirty-label poisoning attack to estimate the upper bound performance of our clean-label poisoning attack.

Original: Table 2 reports the attack performance.
Revised: Table 2 reports the attack performance.

Original: Our results show that the attack performance declines when the attacker’s shadow data has a different distribution from the actual clean dataset.
Revised: Our results show that the attack performance declines when the attacker’s shadow data has a different distribution from the actual clean dataset.

Original: 5.2 Study 2: Impact of b poison
Revised: 5.2 Study 2: Impact of b poison

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: We then study how the size of poisoning samples affects the attack performance.
Revised: We then study how the size of poisoning samples affects the attack performance.

Original: Figure 4 depicts the membership exposure under dirty-label poisoning attacks with different poisoning budget b poison .
Revised: Figure 4 depicts the membership exposure under dirty-label poisoning attacks with different poisoning budget b poison .

Original: Unsurprisingly, as we gradually decrease the poisoning budget, the membership inference AUC score decreases as well.
Revised: Unsurprisingly, as we gradually decrease the poisoning budget, the membership inference AUC score decreases as well.

Original: Yet, it is still possible to achieve a signiﬁcant membership exposure increase even with a small poison budget.
Revised: Yet, it is still possible to achieve a signiﬁcant membership exposure increase even with a small poison budget.

Original: Take ResNet50 for instance, when b poison is 100,
Revised: Take ResNet50 for instance, when b poison is 100,

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: e., the amount of poisoning samples is only 1% of that for the clean samples, we can improve the membership inference AUC score from 0.5971 to 0.7076 on average.
Revised: e., the amount of poisoning samples is only 1% of that for the clean samples, we can improve the membership inference AUC score from 0.5971 to 0.7076 on average.

Original: 5.3 Impact of Fine-tuning
Revised: 5.3 Impact of Fine-tuning

Original: [Figure]
Revised: [Figure]

Original: 02468(b) Poisons 02468(c)
Revised: [Figure] Poisons 02468(c)

Original: Poisons 02468(d)
Revised: Poisons 02468(

Original:  In the transfer learning scenario, there are also chances that model developers ﬁne-tune parameters of feature extractors [47, 14, 36].
Revised: [Figure] In the transfer learning scenario, there are also chances that model developers ﬁne-tune parameters of feature extractors [47, 14, 36].

Original: The ﬁne-tuning operation is supposed to make the transfer learning model better ﬁt the new dataset.
Revised: The ﬁne-tuning operation is supposed to make the transfer learning model better ﬁt the new dataset.

Original: Inevitably, it may affect how the feature extractor extracts latent features from the inputs.
Revised: Inevitably, it may affect how the feature extractor extracts latent features from the inputs.

Original: As a result, our poisoning attack may get affected.
Revised: As a result, our poisoning attack may get affected.

Original: In this part, we explore how the ﬁne-tuning process impacts our attack performance.
Revised: In this part, we explore how the ﬁne-tuning process impacts our attack performance.

Original: In our experiment, we train CIFAR-10 classiﬁers with ﬁne-tuning on the poisoned dataset, with the learning rate set as 1e-5.
Revised: In our experiment, we train CIFAR-10 classiﬁers with ﬁne-tuning on the poisoned dataset, with the learning rate set as 1e-5.

Original: We report the attack performance in Table 3.
Revised: We report the attack performance in Table 3.

Original: It can be seen that, for our dirty-label poisoning attack, we achieve more signiﬁcant membership exposure ampliﬁcations.
Revised: It can be seen that, for our dirty-label poisoning attack, we achieve more signiﬁcant membership exposure ampliﬁcations.

Original: While for our clean-label poisoning attack, the membership exposure changes are nearly negligible.
Revised: While for our clean-label poisoning attack, the membership exposure changes are nearly negligible.

Original: To understand this phenomenon, we investigate how the feature extractor behaves in different cases.
Revised: To understand this phenomenon, we investigate how the feature extractor behaves in different cases.

Original: We visualize the latent features of clean samples and poisoning samples by t-SNE technique [45] in Figure 5.
Revised: We visualize the latent features of clean samples and poisoning samples by t-SNE technique [45] in Figure 5.

Original: We can observe that, for the dirty-label attack in the ﬁne-tuning setting, the poisoning samples distribute closely to the targeted clean samples in the feature space (as shown in Figure 5c.
Revised: We can observe that, for the dirty-label attack in the ﬁne-tuning setting, the poisoning samples distribute closely to the targeted clean samples in the feature space (as shown in Figure 5c.

Original: Besides, for the clean-label attack in the ﬁne-tuning setting, the poisoning samples distribute among each class in the feature space (as shown in Figure 5d).
Revised: Besides, for the clean-label attack in the ﬁne-tuning setting, the poisoning samples distribute among each class in the feature space (as shown in Figure 5d).

Original: It indicates that the more poisoning samples in the target class, the more signiﬁcant membership exposure results we can get.
Revised: It indicates that the more poisoning samples in the target class, the more signiﬁcant membership exposure results we can get.

Original: By comparing Figure 5a and Figure 5b, we can also partially illustrate that dirty-label poisoning tends to achieve better attack performance than the clean-label poisoning.
Revised: By comparing Figure 5a and Figure 5b, we can also partially illustrate that dirty-label poisoning tends to achieve better attack performance than the clean-label poisoning.

Original: This trend is actually consistent with the intuition of our attack.
Revised: This trend is actually consistent with the intuition of our attack.

Original: In fact, this is the case called “adversarial training” by some adversarial example defense literature [44, 35, 49].
Revised: In fact, this is the case called “adversarial training” by some adversarial example defense literature [44, 35, 49].

Original: During adversarial training, model parameters are gradually tuned to eliminate the effects of input perturbations.
Revised: During adversarial training, model parameters are gradually tuned to eliminate the effects of input perturbations.

Original:  Discussion on Potential Countermeasures
Revised:  Discussion on Potential Countermeasures

Original: In general, there are two potential ways to defend our poisoning attacks: One is to detect and ﬁlter out poisoning data, while the other is to reduce membership exposure.
Revised: In general, there are two potential ways to defend our poisoning attacks: One is to detect and ﬁlter out poisoning data, while the other is to reduce membership exposure.

Original: We will investigate the former direction in the future, and in this part, we mainly explore defenses by limiting membership information leakage, including:
Revised: We will investigate the former direction in the future, and in this part, we mainly explore defenses by limiting membership information leakage, including:

Original: • Regularization.
Revised: • Regularization.

Original: Overﬁtting is considered to be one of the major culprits of membership exposure [38, 51, 33].
Revised: Overﬁtting is considered to be one of the major culprits of membership exposure [38, 51, 33].

Original: Therefore, the regularization technique may be feasible to defend against our attacks.
Revised: Therefore, the regularization technique may be feasible to defend against our attacks.

Original: In our experiment, we introduce an L 2 -norm regularizer with a penalty of 0.05 during the model training process.
Revised: In our experiment, we introduce an L 2 -norm regularizer with a penalty of 0.05 during the model training process.

Original: • Early stopping.
Revised: • Early stopping.

Original: Early stopping is another common practice to prevent overﬁtting [21].
Revised: Early stopping is another common practice to prevent overﬁtting [21].

Original: During the training process, for each epoch, we randomly sample out 10% of the training data as validation data and use 90% of other data as training data.
Revised: During the training process, for each epoch, we randomly sample out 10% of the training data as validation data and use 90% of other data as training data.

Original: We monitor the validation loss, and if it does not decrease in three epochs, we stop the training process.
Revised: We monitor the validation loss, and if it does not decrease in three epochs, we stop the training process.

Original: • DP-SGD.
Revised: • DP-SGD.

Original: Differential privacy (DP) provides a rigorous guarantee to limit privacy leakage [12, 13].
Revised: Differential privacy (DP) provides a rigorous guarantee to limit privacy leakage [12, 13].

Original: Recently, privacy-preserving machine learning algorithms under the differential privacy have been proposed [41, 6, 18], among which differentially private stochastic descent (DP-SGD) [4] receives the most attention [20, 5, 28].
Revised: Recently, privacy-preserving machine learning algorithms under the differential privacy have been proposed [41, 6, 18], among which differentially private stochastic descent (DP-SGD) [4] receives the most attention [20, 5, 28].

Original: In our experiment, we utilize the DP-SGD optimizer provided by the TensorFlow Privacy package 4 to implement differential privacy training.
Revised: In our experiment, we utilize the DP-SGD optimizer provided by the TensorFlow Privacy package 4 to implement differential privacy training.

Original: The hyperparameters used in our implementation are summarized in Table 4.
Revised: The hyperparameters used in our implementation are summarized in Table 4.

Original: As reported by the analysis tool provided by TensorFlow Privacy, we achieve (3.25, 10  6 )
Revised: As reported by the analysis tool provided by TensorFlow Privacy, we achieve (3.25, 10  6 )

Original: -differential privacy on the clean model ( |D train | = |D clean | =10,000), while we achieve (3.10, 10  6 )
Revised: -differential privacy on the clean model ( |D train | = |D clean | =10,000), while we achieve (3.10, 10  6 )

Original: -differential privacy on poisoned models ( b poison = 1 , 000 , |D train | =
Revised: -differential privacy on poisoned models ( b poison = 1 , 000 , |D train | =

Original: |D clean [ D poison | =11,000).
Revised: |D clean [ D poison | =11,000).

Original: We evaluate the three potential countermeasures on the CIFAR-10 dataset and summarize the results in Table 5.
Revised: We evaluate the three potential countermeasures on the CIFAR-10 dataset and summarize the results in Table 5.

Original: By comparing the membership exposure between unprotected (Table 1) and protected models (Table 5), the three defenses can help weaken the membership exposure ampliﬁcation effect by our attacks.
Revised: By comparing the membership exposure between unprotected (Table 1) and protected models (Table 5), the three defenses can help weaken the membership exposure ampliﬁcation effect by our attacks.

Original: Besides, in our experiment, we observe that the regularization technique has the best model utility-privacy trade-off.
Revised: Besides, in our experiment, we observe that the regularization technique has the best model utility-privacy trade-off.

Original: The regularization technique achieves a relatively high model testing accuracy and keeps the AUC scores at lower levels.
Revised: The regularization technique achieves a relatively high model testing accuracy and keeps the AUC scores at lower levels.

Original: Hyperparameter
Revised: Hyperparameter

Original: Value
Revised: Value

Original: LEARNING RATE
Revised: LEARNING RATE

Original: [Table]
Revised: [Table]

Original: NOISE MULTIPLIER 1.
Revised: NOISE MULTIPLIER 1.

Original: MAX L 2 - NORM OF GRADIENTS 1.
Revised: MAX L 2 - NORM OF GRADIENTS 1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we demonstrate the feasibility of exploiting data poisoning to amplify the membership exposure of the training dataset.
Revised: In this paper, we demonstrate the feasibility of exploiting data poisoning to amplify the membership exposure of the training dataset.

Original: We present one signiﬁcantly increase the precision of the membership inference attack on classes with limited negative inﬂuences on the model’s test-time performance.
Revised: We present one signiﬁcantly increase the precision of the membership inference attack on classes with limited negative inﬂuences on the model’s test-time performance.

Original: We also conduct extensive evaluations to study how different factors may affect attack performance.
Revised: We also conduct extensive evaluations to study how different factors may affect attack performance.

Original: Our ﬁndings uncover a new challenge to modern machine learning ecosystems.
Revised: Our ﬁndings uncover a new challenge to modern machine learning ecosystems.

Original: Data from the open world may not only affect the model performance but also raise real threats to private training data.
Revised: Data from the open world may not only affect the model performance but also raise real threats to private training data.

Original: Even worse, we show it is possible to mount clean-label attacks to evade human moderation.
Revised: Even worse, we show it is possible to mount clean-label attacks to evade human moderation.

Original: Our research is thus a call to action.
Revised: Our research is thus a call to action.

Original: One primary limitation of our clean-label attacks is that we assume the attacker has knowledge of the feature extractor.
Revised: One primary limitation of our clean-label attacks is that we assume the attacker has knowledge of the feature extractor.

Original: How to create clean-label poisons in a black-box manner remains our future work.
Revised: How to create clean-label poisons in a black-box manner remains our future work.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: We thank the anonymous reviewers for their constructive comments.
Revised: We thank the anonymous reviewers for their constructive comments.

Original: This work is supported by the
Revised: This work is supported by the

Original: National Key Research and Development Program of China (2020AAA0107702), National Natural
Revised: National Key Research and Development Program of China (2020AAA0107702), National Natural

Original: Science Foundation of China (U21B2018, 62161160337, 62132011), Shaanxi Province Key Industry
Revised: Science Foundation of China (U21B2018, 62161160337, 62132011), Shaanxi Province Key Industry

Original: Innovation Program (2021ZDLGY01-02), the Research Grants Council of Hong Kong under Grants
Revised: Innovation Program (2021ZDLGY01-02), the Research Grants Council of Hong Kong under Grants

Original: N_CityU139/21, R6021-20F,
Revised: N_CityU139/21, R6021-20F,

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We will submit the code, link to data we used, and instructions in the supplemental material.
Revised: [Yes] We will submit the code, link to data we used, and instructions in the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] We reset the random seed (54321) at the beginning of building clean models and poisoned models.
Revised: [No] We reset the random seed (54321) at the beginning of building clean models and poisoned models.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We cite the data and models we used in Section 4.1.
Revised: [Yes] We cite the data and models we used in Section 4.1.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] We did not use any new assets.
Revised: [No] We did not use any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [No] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [No] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [No]
Revised: [No]

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]