Original: Amplifying Membership Exposure via Data Poisoning
Revised: Amplifying Membership Exposure via Data Poisoning

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: As in-the-wild data are increasingly involved in the training stage, machine learningapplications become more susceptible to data poisoning attacks.
Revised: As in-the-wild data are increasingly involved in the training stage, machine learning applications become more susceptible to data poisoning attacks.

Original: Such attackstypically lead to test-time accuracy degradation or controlled misprediction.
Revised: Such attacks typically lead to test-time accuracy degradation or controlled misprediction.

Original: In thispaper, we investigate the third type of exploitation of data poisoning - increasingthe risks of privacy leakage of benign training samples.
Revised: In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples.

Original: To this end, we demonstratea set of data poisoning attacks to amplify the membership exposure of the targetedclass.
Revised: To this end, we demonstrate a set of data poisoning attacks to amplify the membership exposure of the targeted class.

Original: We first propose a generic dirty-label attack for supervised classificationalgorithms.
Revised: We first propose a generic dirty-label attack for supervised classification algorithms.

Original: We then propose an optimization-based clean-label attack in thetransfer learning scenario, whereby the poisoning samples are correctly labeled andlook “natural” to evade human moderation.
Revised: We then propose an optimization-based clean-label attack in the transfer learning scenario, whereby the poisoning samples are correctly labeled and look “natural” to evade human moderation.

Original: We extensively evaluate our attackson computer vision benchmarks.
Revised: We extensively evaluate our attacks on computer vision benchmarks.

Original: Our results show that the proposed attacks cansubstantially increase the membership inference precision with minimum overalltest-time model performance degradation
Revised: Our results show that the proposed attacks can substantially increase the membership inference precision with minimum overall test-time model performance degradation.

Revised: To mitigate the potential negative impacts of our attacks, we also investigate feasible countermeasures.

Original:  Introduction
Revised:  Introduction

Original: Training data are the most critical ingredients of machine learning, which are sometimes regarded as anew type of fuel in the era of artificial intelligence [21].
Revised: Training data are the most critical ingredients of machine learning, which are sometimes regarded as a new type of fuel in the era of artificial intelligence [30].

Original: Over the last decade, training data collectionand preservation have received growing concerns [20, 6], among which two focal agendas arise.
Revised: Over the last decade, training data collection and preservation have received growing concerns [29, 10], among which two focal agendas arise.

Original: One is data corruption caused by data poisoning attacks, which pose serious threats to training dataintegrity .
Revised: One is data corruption caused by data poisoning attacks, which pose serious threats to training data integrity .

Original: Data poisoning attacks exploit the common practice to (usually automatically) collecttraining data in the wild, e.g., from the Internet.
Revised: Data poisoning attacks exploit the common practice to (usually automatically) collect training data in the wild, e.g., from the Internet.

Original: This practice opens doors for attackers to injectmalicious data at the training time to manipulate model behaviors at the test time.
Revised: This practice opens doors for attackers to inject malicious data at the training time to manipulate model behaviors at the test time.

Original: Consequently,models trained with poisoned training data suffer from either accuracy degradation [4, 13], targetedmisclassification [25, 44], or backdoor implantation [23, 41].
Revised: Consequently, models trained with poisoned training data suffer from either accuracy degradation [7, 19], targeted misclassification [34, 55] or backdoor implantation [32, 50].

Original: The other is data leakage caused by privacy inference attacks, which mainly violate training dataconfidentiality .
Revised: The other is data leakage caused by privacy inference attacks, which mainly violate training data confidentiality .

Original: Ideally, a machine learning model learns generalizable knowledge of the trainingdata, rather than details of specific data points.
Revised: Ideally, a machine learning model learns generalizable knowledge of the training data, rather than details of specific data points.

Original: However, previous studies show that conventionallearning algorithms can unintentionally remember sensitive information.
Revised: However, previous studies show that conventional learning algorithms can unintentionally remember sensitive information.

Original: Such information can laterbe inferred by various privacy inference attacks, such as membership inference attacks [29, 24].
Revised: Such information can later be inferred by various privacy inference attacks, such as membership inference attacks [38, 33].

Original: Essentially, data poisoning is a training time attack while privacy inference is performed at the testingtime.
Revised: Essentially, data poisoning is a training time attack, while privacy inference is performed at the testing time.

Original: Despite intensive research efforts, most existing works study them separately.
Revised: Despite intensive research efforts, most existing works study them separately.

Original: It remains unclear whether these two attacks can be integrated to cause more severe damage to machine learning models.
Revised: It remains unclear whether these two attacks can be integrated to cause more severe damage to machine learning models.

Original: In this paper, we aim to advance the research frontier on the connection between data integrityand confidentiality.
Revised: In this paper, we aim to advance the research frontier on the connection between data integrity and confidentiality.

Original: To this end, we propose a set of poisoning attacks to increase the precision of membership inference attacks, which is widely adopted as a standard tool to measure privacy leakage in statistical analysis [31, 10].
Revised: To this end, we propose a set of poisoning attacks to increase the precision of membership inference attacks, which is widely adopted as a standard tool to measure privacy leakage in statistical analysis [40, 15].

Original: In particular, we first show a simple but effective dirty-label poisoningattack that is generic for supervised classification applications.
Revised: In particular, we first show a simple but effective dirty-label poisoning attack that is generic for supervised classification applications.

Original: We then demonstrate a clean-labelpoisoning attack applicable to the transfer learning scenario.
Revised: We then demonstrate a clean-label poisoning attack applicable to the transfer learning scenario.

Original: It has two significant advantages: (1) norequirements on the labeling process and (2) natural appearances to human moderation.
Revised: It has two significant advantages: (1) no requirements on the labeling process and (2) natural appearances to human moderation.

Original: Consequently,our proposed clean-label poisoning attack challenges the case where automatically crawling datafrom the Internet has become a common practice.
Revised: Consequently, our proposed clean-label poisoning attack challenges the case where automatically crawling data from the Internet has become a common practice.

Original: Our contributions are as follows: (1) We reveal an underexplored data poisoning attack threatening thetraining data privacy.
Revised: Our contributions are as follows: (1) We reveal an underexplored data poisoning attack threatening the training data privacy.

Original: In particular, we demonstrate how to amplify membership inference exposureof a specific class with only slight impacts on the model performance.
Revised: In particular, we demonstrate how to amplify membership inference exposure of a specific class with only slight impacts on the model performance.

Original:  In generic supervisedlearning settings, we introduce a naïve dirty-label poisoning attack by modifying the labels of thepoisoning samples.
Revised: (2) In generic supervised learning settings, we introduce a naïve dirty-label poisoning attack by modifying the labels of the poisoning samples.

Original: In the transfer learning setting, we propose an optimization-based clean-labelpoisoning attack, wherein for each poisoning sample, we make imperceptible modifications withoutchanging its label.
Revised: In the transfer learning setting, we propose an optimization-based clean-label poisoning attack, wherein for each poisoning sample, we make imperceptible modifications without changing its label.

Original: It presents a more practical attack example since it does not require control overthe labeling process.
Revised: It presents a more practical attack example since it does not require control over the labeling process.

Original: It is also a more stealthy attack as the contents of poisoning instances “looksimilar” to the natural ones.
Revised: It is also a more stealthy attack as the contents of poisoning instances “look similar” to the natural ones.

Original: (4) We conduct extensive empirical studies to investigate the impacts ofvarious factors, including datasets, architectures, poisoning budgets, and learning setups 1 .
Revised: (4) We conduct extensive empirical studies to investigate the impacts of various factors, including datasets, architectures, poisoning budgets, and learning setups.

Original: Our resultsshow that our proposed poisoning attacks are able to increase the membership exposure obviouslywith just limited influences on the victim model’s performance on testing samples.
Revised: Our results show that our proposed poisoning attacks are able to increase the membership exposure obviously with just limited influences on the victim model’s performance on testing samples.

Revised: At the end, we also consider several potential countermeasures and investigate their effectiveness.

Original:  Background
Revised:  Background

Original: In this work, we focus on deep supervised classification in the computer vision field.
Revised: In this work, we focus on deep supervised classification in the computer vision field.

Original: That is, given atraining dataset D train composed of input-label pairs ( x, y ) , a deep classifier f ( x ; θ ) with model parameters θ , and a learning objective L ( f ( x ; θ ) , y ) , the learning process aims to find a set of model parameters θ ∗ to minimize the learning objective on D train , i.e., θ ∗ = arg min θ[Equation] f
Revised: That is, given a training dataset D train composed of input-label pairs ( x, y ) , a deep classifier f ( x ; θ ) with model parameters θ , and a learning objective L ( f ( x ; θ ) , y ) , the learning process aims to find a set of model parameters θ ∗ to minimize the learning objective on D train , i.e., θ ∗ = arg min θ

Original: [Equation]
Revised: [Equation]

Original: 1 Data Poisoning
Revised: 2.1 Data Poisoning

Original: In a poisoning attack, an attacker crafts and injects a set of malicious training samples D poison (i.e.,poisoning dataset ) into the benign dataset D clean (i.e., clean dataset ).
Revised: In a poisoning attack, an attacker crafts and injects a set of malicious training samples D poison (i.e., poisoning dataset ) into the benign dataset D clean (i.e., clean dataset ).

Original: In the training stage, the modelholder executes the machine learning algorithm on the full poisoned dataset D train =
Revised: In the training stage, the model holder executes the machine learning algorithm on the full poisoned dataset D train =

Original: D clean ∪ D poisonto obtain the trained model f
Revised: D clean ∪ D poison to obtain the trained model f

Original: ( x ; θ ∗ ) .
Revised: ( x ; θ ∗ ) .

Original: In the inference stage, f ( x ; θ ∗ ) tends to show unexpectedbehaviors on targeted inputs ( x, y ) ∈ D target .
Revised: In the inference stage, f ( x ; θ ∗ ) tends to show unexpected behaviors on targeted inputs ( x, y ) ∈ D target .

Original: Most existing literature poses the poisoning attack as abi-level optimization problem [5, 11]:
Revised: Most existing literature poses the poisoning attack as a bi-level optimization problem [8, 16]:

Original: [Equation]
Revised: [Equation]

Original: where A is the adversarial objective of the poisoning attack.
Revised: where A is the adversarial objective of the poisoning attack.

Original: Typically, there are three adversarialobjectives: (1) in the accuracy degradation case, D target contains all testing samples andA =
Revised: Typically, there are three adversarial objectives: (1) in the accuracy degradation case, D target contains all testing samples and

Original:  −L ( f ( x ; θ ∗ ) , y ) ;
Revised: A = −L ( f ( x ; θ ∗ ) , y ) ;

Original: (2) in the targeted misclassification case, D target contains samples expected to bemisclassified into the target class t and A =
Revised: (2) in the targeted misclassification case, D target contains samples expected to be misclassified into the target class t and A =

Original: L ( f ( x ; θ ∗ ) , t ) ; (3) in the backdoor implantation case,
Revised: L ( f ( x ; θ ∗ ) , t ) ; (3) in the backdoor implantation case,

Original: D target involves samples with a trigger δ , where inputs embedded with the trigger are to be classifiedinto the target class t and A =
Revised: D target involves samples with a trigger δ , where inputs embedded with the trigger are to be classified into the target class t and A =

Original: L ( f ( x ⊕ δ ; θ ∗ ) , t ) .
Revised: L ( f ( x ⊕ δ ; θ ∗ ) , t ) .

Original: Based on the attacker’s capability, poisoning attacks can be further grouped into two categories:
Revised: Based on the attacker’s capability, poisoning attacks can be further grouped into two categories:

Original: • Dirty-label poisoning.
Revised: • Dirty-label poisoning.

Original: Most classical poisoning attacks require the control of the labelingprocess, where the attacker is allowed to modify the labels in D poison [13, 39, 33].
Revised: Most classical poisoning attacks require the control of the labeling process, where the attacker is allowed to modify the labels in D poison [19, 48, 42].

Original: Suchattack is called dirty-label poisoning .
Revised: Such an attack is called dirty-label poisoning .

Original: Despite promising attack performance, dirty-labelpoisoning becomes impractical in supervised machine learning scenarios for two mainreasons.
Revised: Despite promising attack performance, dirty-label poisoning becomes impractical in supervised machine learning scenarios for two main reasons.

Original: First, in reality, it is a common practice that only unlabeled data (e.g., images andvideos) are scraped by crawlers and then labeled by human moderators.
Revised: First, in reality, it is a common practice that only unlabeled data (e.g., images and videos) are scraped by crawlers and then labeled by human moderators.

Original: Second, machine learning developers usually utilize anomaly detection algorithms to filter out wrongly labeled samples.
Revised: Second, machine learning developers usually utilize anomaly detection algorithms to filter out wrongly labeled samples.

Original: Therefore, there exists a minimal chance that the data samples with modified labelsare preserved in the dataset.
Revised: Therefore, there exists a minimal chance that the data samples with modified labels are preserved in the dataset.

Original: • Clean-label poisoning.
Revised: • Clean-label poisoning.

Original: To overcome the shortcomings of dirty-label poisoning, recentstudies propose clean-label poisoning attacks [25, 43, 44].
Revised: To overcome the shortcomings of dirty-label poisoning, recent studies propose clean-label poisoning attacks [34, 53, 55].

Original: A clean-label poisoning attacksatisfies two properties.
Revised: A clean-label poisoning attack satisfies two properties.

Original: First, the labels of poisoning samples remain unchanged in the poisoning process.
Revised: First, the labels of poisoning samples remain unchanged in the poisoning process.

Original: Second, to be inconspicuous, each poisoning sample x p visually resemblesa natural sample x n , which is always constrained by a L p -norm distance ∥ x p − x n ∥ p < ϵ .Our
Revised: Second, to be inconspicuous, each poisoning sample x p visually resembles a natural sample x n , which is always constrained by a L p -norm distance ∥ x p − x n ∥ p < ϵ .

Original:  setup.
Revised: Our setup.

Original: In this work, we investigate data poisoning under a newly identified and underexplored adversarial objective: privacy leakage amplification .
Revised: In this work, we investigate data poisoning under a newly identified and underexplored adversarial objective: privacy leakage amplification .

Original: In particular, we focus on one of the mostrepresentative privacy inference attacks: membership inference attack [29, 24].
Revised: In particular, we focus on one of the most representative privacy inference attacks: membership inference attack [38, 33].

Original: We start from thedirty-label poisoning setup, which establishes a baseline and cornerstone for more advanced attacks.Then
Revised: We start from the dirty-label poisoning setup, which establishes a baseline and cornerstone for more advanced attacks.

Original: , we investigate attack methods in the clean-label poisoning setup to improve practicability.
Revised: Then, we investigate attack methods in the clean-label poisoning setup to improve practicability.

Original: Note

Original:  that there are concurrent works that propose to use poisoning attacks to cause privacy leakage [18, 34]
Revised: Note that there are concurrent works that propose to use poisoning attacks to cause privacy leakage [26, 43].

Original: 2.2 Membership Inference Attack
Revised: 2.2 Membership Inference Attack

Original: In a membership inference attack, an attacker aims to infer whether a specific sample ( x, y ) belongsto the training dataset D train at the test time [29, 24, 16].
Revised: In a membership inference attack, an attacker aims to infer whether a specific sample ( x, y ) belongs to the training dataset D train at the test time [38, 33, 24].

Original: Unintended membership exposure causescatastrophic privacy loss for individuals.
Revised: Unintended membership exposure causes catastrophic privacy loss for individuals.

Original: For example, in the real world, a data sample x can be aclinical record or an individual.
Revised: For example, in the real world, a data sample x can be a clinical record or an individual.

Original: Membership inference attacks enable the attackers to infer whetherthis clinical record or individual has been used to train a model associated with a certain disease.
Revised: Membership inference attacks enable the attackers to infer whether this clinical record or individual has been used to train a model associated with a certain disease.

Original: Assuch, these attacks are widely adopted as basic metrics to quantifythe privacy exposure in statisticaldata analysis algorithms [8, 15].
Revised: As such, these attacks are widely adopted as basic metrics to quantify privacy exposure in statistical data analysis algorithms [13, 23].

Original: Henceforth, we use membership inference attacks to demonstratehow to exploit data poisoning to amplify privacy leakage in this paper.
Revised: Henceforth, we use membership inference attacks to demonstrate how to exploit data poisoning to amplify privacy leakage in this paper.

Original: Based on the attacker’s capability, membership inference attacks can be grouped into two categories:
Revised: Based on the attacker’s capability, membership inference attacks can be grouped into two categories:

Original: • Black-box membership inference.
Revised: • Black-box membership inference.

Original: In this case, the attacker distinguishes members and non-members only using model outputs [29, 28, 12].
Revised: In this case, the attacker distinguishes members and non-members only using model outputs [38, 37, 17].

Original: This case is generic to most machine learning contexts.
Revised: This case is generic to most machine learning contexts.

Original: There are two attack strategies in general.
Revised: There are two attack strategies in general.

Original: The first strategy is model- based [29], where the attacker builds multiple shadow models to mimic the victim model, then utilizes them to construct a dataset with member/non-member labels, and finally trains a binary classifier to predict member/non-member.
Revised: The first strategy is modelbased [38], where the attacker builds multiple shadow models to mimic the victim model, then utilizes them to construct a dataset with member/non-member labels, and finally trains a binary classifier to predict member/non-member.

Original: The second strategy is metric- based [24, 30], where the attacker compares a designed metric M mem , such as correctness or entropy, with a predefined threshold τ to infer if a sample belongs to the training dataset.
Revised: The second strategy is metricbased [33, 39], where the attacker compares a designed metric M mem , such as correctness or entropy, with a predefined threshold τ to infer if a sample belongs to the training dataset.

Original: • White-box membership inference.
Revised: • White-box membership inference.

Original: In this case, model parameters θ ∗ or even intermediate training information such as gradients ∂
Revised: In this case, model parameters θ ∗ or even intermediate training information such as gradients ∂

Original: L ∂θ are observable by the attacker [19, 22, 14].
Revised: L ∂θ are observable by the attacker [27, 31, 22].

Original: Such capability provides additional information supporting inference attacks, which is usually achievable in collaborative learning settings.
Revised: Such capability provides additional information supporting inference attacks, which is usually achievable in collaborative learning settings.

Original: Most white-box attacks are model-based, as the attacker needs to access the internals of deep models to extract model-specific features
Revised: Most white-box attacks are model-based, as the attacker needs to access the internals of deep models to extract model-specific features.

Original: Our setup.
Revised: Our setup.

Original: Our evaluation is carried out in the metric-based black-box membership inferencesetting.
Revised: Our evaluation is carried out in the metric-based black-box membership inference setting.

Original: We adopt the following metric proposed by Song et al.
Revised: We adopt the following metric proposed by Song et al.

Original: [31] tohelp measure the membershipexposure 2 :
Revised: [40] to measure the membership exposure 2 :

Original: [Equation]
Revised: [Equation]

Original: where f ( x ) j refers to the confidence value of label j .
Revised: where f ( x ) j refers to the confidence value of label j .

Original: Equation 2 simultaneously considers thecorrectness and entropy metric.
Revised: Equation 2 simultaneously considers the correctness and entropy metric.

Original: In the training stage, the learning algorithm continuously fits thetraining samples, by decreasing the entropy loss (i.e., the learning objective) and increasing theconfidence score of the correct label.
Revised: In the training stage, the learning algorithm continuously fits the training samples, by decreasing the entropy loss (i.e., the learning objective) and increasing the confidence score of the correct label.

Original: As a result, a member is likely to produce a higher M mem than anon-member.
Revised: As a result, a member is likely to produce a higher M mem than a non-member.

Original: In our evaluation, we  adopt the AUC  score to measure membership exposure when thethreshold τ varies.
Revised: To gain a holistic view of the membership exposure, we calculate the true positive rate (TPR) and false positive rate (FPR) of the membership inference attack by varying the threshold τ , and plot the ROC curve. Then We adopt the AUC (area under the ROC curve) score to measure membership exposure. In general, a higher AUC score means a higher risk of membership exposure, as we can find a threshold τ with high TPR and low FPR. It is notable that the AUC score is an average-case metric, which hardly effectively reflects the worst-case privacy [9].

Revised: Therefore, we also report the TPR when the FPR is low (1% in our case), which works as a compensation for the AUC metric to indicate the worst-case privacy.

Original:  2.3 Threat Model
Revised: x ) to represent the trained model f ( x ; θ ∗ ) . We also evaluate our poisoning attacks with a stronger membership inference attack [9] on the CIFAR-10 dataset. The experimental results show that our poisoning attacks can also help improve the membership 2.3 Threat Model

Original: Before diving into our attack design, we first elaborate on the threat model considered in this paper.
Revised: Before diving into our attack design, we first elaborate on the threat model considered in this paper.

Original: Attack Goals.
Revised: Attack Goals.

Original: The first goal of the attacker is increasing the chance of leaking the membershipof training samples within a targeted class .
Revised: The first goal of the attacker is increasing the chance of leaking the membership of training samples within a targeted class .

Original: At the same time, the attacker attempts to make thepoisoning attack as stealthy as possible.
Revised: At the same time, the attacker attempts to make the poisoning attack as stealthy as possible.

Original: As such, the second goal of the attacker is generatingpoisoning samples that have limited impact on the model performance for untargeted classes and areindistinguishable to natural samples .
Revised: As such, the second goal of the attacker is generating poisoning samples that have limited impacts on the model performance for untargeted classes and are indistinguishable to natural samples .

Original: Attacker Capabilities.
Revised: Attacker Capabilities.

Original: We assume the attacker has the basic capabilities set up by existing datapoisoning and membership inference attack games.
Revised: We assume the attacker has the basic capabilities set up by existing data poisoning and membership inference attack games.

Original: The attacker can craft poisoning samples andinject them into the victim’s clean dataset D clean .
Revised: The attacker can craft poisoning samples and inject them into the victim’s clean dataset D clean .

Original: However, there exists a poisoning budget b poison tolimit the amount of training samples ( |D poison | ≤ b poison ≪
Revised: However, there exists a poisoning budget b poison to limit the amount of training samples ( |D poison | ≤ b poison ≪

Original: |D clean | ).
Revised: |D clean | ).

Original: We assume the attacker owns ashadow dataset D shadow to craft poisoning samples, which contains natural samples from the samedistribution with D clean .
Revised: We assume the attacker owns a shadow dataset D shadow to craft poisoning samples, which contains natural samples from the same distribution with D clean .

Original: Moreover, the attacker cannot modify poisoning data labels in the clean-labelpoisoning setting.
Revised: Moreover, the attacker cannot modify poisoning data labels in the clean-label poisoning setting.

Original: But we assume the attacker knows the feature extractor (i.e., an encoder) usedby the victim model.
Revised: But we assume the attacker knows the feature extractor (i.e., an encoder) used by the victim model.

Original: This assumption is practical since developing models with public pretrainedfeature extractors is a common practice in existing clean-label poisoning literature [25, 43].
Revised: This assumption is practical since developing models with public pretrained feature extractors is a common practice in existing clean-label poisoning literature [34, 53].

Original: Remark.
Revised: Remark.

Original: In our setup, the clean dataset D clean is unknown by the attacker, which is different frommost existing data poisoning games [4, 13, 18].
Revised: In our setup, the clean dataset D clean is unknown by the attacker, which is different from most existing data poisoning games [7, 19, 26].

Original:  Attack Methodology
Revised:  Attack Methodology

Original: Although Equation 1 establishes a generic framework for poisoning attack, it is unsuitable for ourcase.
Revised: Although Equation 1 establishes a generic framework for poisoning attack, it is unsuitable for our case.

Original: First, classic solutions for the optimization problem posed by Equation 1 require differentiationto the inner loss minimizer, which is intractable to models with high complexity.
Revised: First, classic solutions for the optimization problem posed by Equation 1 require differentiation to the inner loss minimizer, which is intractable to models with high complexity.

Original: Second, althoughmeta-learning algorithms have been proposed to solve the bi-level optimization problem in the deeplearning setting [11, 42, 32], they lead to huge computing costs, limiting the practicability of theattack.
Revised: Second, although meta-learning algorithms have been proposed to solve the bi-level optimization problem in the deep learning setting [16, 52, 54], they lead to huge computing costs, limiting the practicability of the attack.

Original: Third, in our threat model, D clean is unobservable, making Equation 1 no longer applicable toour attack.
Revised: Third, in our threat model, D clean is unobservable, making Equation 1 no longer applicable to our attack.

Original: Facing these obstacles, we explore heuristic strategies to achieve the attack 
Revised: Facing these obstacles, we explore heuristic strategies to achieve the attack goals.

Original: [Figure]

Original: 3.1 Dirty-label Poisoning Attack: A Naïve Approach
Revised: 3.1 Dirty-label Poisoning Attack: A Naïve Approach

Original: [Figure]
Revised: [Figure]

Original: We start our investigation in the dirty-label poisoning setting.
Revised: We start our investigation in the dirty-label poisoning setting.

Original: This case has more limitations inthe real world as discussed in Section 2.1.
Revised: This case has more limitations in the real world as discussed in Section 2.1.

Original: But itrelaxes the attack constraints and helps to verifythe feasibility of our attack strategy.
Revised: But it relaxes the attack constraints and helps to verify the feasibility of our attack strategy.

Original: The key to amplifying membership exposure isto cause overfitting in the targeted class.
Revised: The key to amplifying membership exposure is to cause overfitting in the targeted class.

Original: Wepropose the following attack strategy:
Revised: We propose the following attack strategy:

Original: Our dirty-label poisoning attack.
Revised: Our dirty-label poisoning attack.

Original: Given thelabel of the target group t , the shadow dataset
Revised: Given the label of the target group t , the shadow dataset

Original: D shadow , and a poisoning budget b poison , the poisoning dataset is constructed by the followingsteps: (1) select all samples ( x, t ) ∈
Revised: D shadow , and a poisoning budget b poison , the poi- soning dataset is constructed by the following steps: (1) select all samples ( x, t ) ∈

Original: D shadowwith label t ; (2) for each sample ( x, t ) , randomly change the label to another class i ̸
Revised: D shadow with label t ; (2) for each sample ( x, t ) , randomly change the label to another class i ̸

Original: = t ; (3) preserveat most b poison samples ( x, i ) to compose D poison .
Revised: = t ; (3) preserve at most b poison samples ( x, i ) to compose D poison .

Original: Intuitively, only learning general concepts/features is insufficient to discriminate the clean samples ofclass t from poisoning samples, since in fact, they share similar features.
Revised: Intuitively, only learning general concepts/features is insufficient to discriminate the clean samples of class t from poisoning samples, since in fact, they share similar features.

Original: To minimize the trainingloss, the victim model has to learn more specific features of each sample.
Revised: To minimize the training loss, the victim model has to learn more specific features of each sample.

Original: As conceptually illustratedby Figure 1, the generalization performance on class t tend to degrade.
Revised: As conceptually illustrated by Figure 1, the generalization performance on class t tend to degrade.

Original: Figure 2 gives an example to show how our poisoning attack improves the membership exposure of the airplane category for an 
Revised: Figure 2 gives an example to show how our poisoning attack improves the membership exposure of the airplane category for an inference accuracy by a stronger membership inference attacker.

Revised: We recommend interested readers find more results paper, experiments weak lower

Original: InceptionV3-based CIFAR-10 classifier.
Revised: InceptionV3-based CIFAR-10 classifier.

Original: In this example, we assume the numbers of members andnon-members are both 10,000 and b poison =1,000.[Figure] T r u e P o s i t i v e R a t e
Revised: In this example, we assume the numbers of members and non-members are both 10,000 and b poison =1,000.

Original: [Figure]
Revised: [Figure]

Original: Remark.
Revised: Remark.

Original: We do not pose extra assumptions to thetraining process in this case.
Revised: We do not pose extra assumptions to the training process in this case.

Original: Our dirty-label poisoning attack is generic to all supervised learningscenarios, including end-to-end learning and transferlearning.
Revised: Our dirty-label poisoning attack is generic to all supervised learning scenarios, including end-to-end learning and transfer learning.

Original: Similar to the concurrent work [34], we alsouse the classical label flipping strategy [4] to carryout the dirty-label attack.
Revised: Similar to the concurrent work [43], we also use the classical label flipping strategy [7] to carry out the dirty-label attack.

Original: 3.2 Clean-label 
Revised: 3.2 Clean-label Poisoning Attack: A Stealthy Approach

Original: Poisoning Attack: A Stealthy Approach

Original: In this part, we aim to improve our poisoning attack toamplify membership exposure while attaining stealthiness.
Revised: In this part, we aim to improve our poisoning attack to amplify membership exposure while attaining stealthiness.

Original: Following the basic setup in [25], we considerthe transfer learning setting.
Revised: Following the basic setup in [34], we consider the transfer learning setting.

Original: In transfer learning, weassume the victim model is composed of two parts: apretrained feature extractor g ( · ) to extract high-levelfeatures from the input x and a newly-trained classifier c ( · ) to predict labels.
Revised: In transfer learning, we assume the victim model is composed of two parts: a pretrained feature extractor g ( · ) to extract high-level features from the input x and a newly-trained classifier c ( · ) to predict labels.

Original: The key to our clean-label poisoning is to exploit one vulnerability of deep feature extractors: aslight change in the input space may cause a significant change in the feature space.
Revised: The key to our clean-label poisoning is to exploit one vulnerability of deep feature extractors: a slight change in the input space may cause a significant change in the feature space.

Original: We propose thefollowing attack strategy.
Revised: We propose the following attack strategy.

Original: Our clean-label poisoning attack.
Revised: Our clean-label poisoning attack.

Original: Given the label of the target class t , the shadow dataset D shadow ,a poisoning budget b poison , and the adopted feature extractor g , the poisoning dataset is constructed bythe following steps: (1) from D shadow , select a base sample ( x base , t ) with label t and a sample ( x, y )where y ̸ = t ; (2) find a x ∗ that is close to x in the input space ( ∥ x ∗ − x ∥ p ≤ ϵ ) and close to x base inthe feature space ( g ( x ∗ )
Revised: Given the label of the target class t , the shadow dataset D shadow , a poisoning budget b poison , and the adopted feature extractor g , the poisoning dataset is constructed by the following steps: (1) from D shadow , select a base sample ( x base , t ) with label t and a sample ( x, y ) where y ̸ = t ; (2) find a x ∗ that is close to x in the input space ( ∥ x ∗ − x ∥ p ≤ ϵ ) and close to x base in the feature space ( g ( x ∗ )

Original: ≈ g ( x base ) ), and insert ( x ∗ , y ) to D poison ; (3) repeat step (1) and (2) until|D poison | =
Revised: ≈ g ( x base ) ), and insert ( x ∗ , y ) to D poison ; (3) repeat step (1) and (2) until |D poison | =

Original: b poison .
Revised: b poison .

Original: The intuition of our clean-label poisoning attack is straightforward: we actually mount a dirty-labelpoisoning attack in the feature space.
Revised: The intuition of our clean-label poisoning attack is straightforward: we actually mount a dirty-label poisoning attack in the feature space.

Original: That is the poisoning sample ( x ∗ , y ) is close to  clean samplesin class t  in the feature space but labeled as y .
Revised: That is, the poisoning sample ( x ∗ , y ) is close to ( x base , t ) , a clean sample from class t , in the feature space but labeled as y .

Original: We formalize step (2) as a constrained optimizationproblem:
Revised: We formalize step (2) as a constrained optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where X refers to the input space.
Revised: where X refers to the input space.

Original: For instance, for a normalized RGB image input, a valid pixelvalue in one color channel should be within [0 , 1] .
Revised: For instance, for a normalized RGB image input, a valid pixel value in one color channel should be within [0 , 1] .

Original: For ease of optimization, we adopt L 2 -norm tomeasure the distance in the feature space.
Revised: For ease of optimization, we adopt L 2 -norm to measure the distance in the feature space.

Original: Following the convention of adversarial attack work, weadopt L ∞ to measure the distance in the input space.
Revised: Following the convention of adversarial attack work, we adopt L ∞ to measure the distance in the input space.

Original: In our case, the constraint can be expressed asx ′ ∈ [ x min , x max ] , where x min = max(0 , xbase − ϵ ) , x max = min(1 , xbase + ϵ ) .
Revised: In our case, the constraint can be expressed as x ′ ∈ [ x min , x max ] , where x min = max(0 , x − ϵ ) , x max = min(1 , x + ϵ ) .

Original: Optimization.
Revised: Optimization.

Original: We adopt the variable substitution method introduced by [7] to solve Equation 3.
Revised: We adopt the variable substitution method introduced by [11] to solve Equation 3.

Original: Weintroduce a new variable w and let
Revised: We introduce a new variable w and let

Original: [Equation]
Revised: [Equation]

Original: Based on the fact that tanh( w ) ∈ [ − 1 , 1] , we can prove that the constraint x ′ ∈ [ x min , x max ] holdsin Equation 4.
Revised: Based on the fact that tanh( w ) ∈ [ − 1 , 1] , we can prove that the constraint x ′ ∈ [ x min , x max ] holds in Equation 4.

Original: Plugging Equation 4 into Equation 3, we can obtainw ∗ =
Revised: Plugging Equation 4 into Equation 3, we can obtain

Original: arg min w ∥ g (0 . 5 ∗ (tanh( w ) + 1)( x max − x min ) + x min ) − g ( x base ) ∥ 2 , and x ∗ = 0 . 
Revised: [Equation] and x ∗ = 0 . [Equation]

Original: 5(tanh( w ∗ ) + 1)( x max − x min ) + x min (5) which poses an unconstrained optimization problem that can be simply solved by mainstream gradientoptimization methods.
Revised: which poses an unconstrained optimization problem that can be simply solved by mainstream gradient optimization methods.

Original: In our implementation, we first initialize variable w with x base by Equation 4and iterate the optimization process 1,000 times, using an Adam optimizer with the learning rate 0.01.
Revised: In our implementation, we first initialize variable w with x base by Equationand iterate the optimization process 1,000 times, using an Adam optimizer with the learning rate 0.01.

Original: Figure 3 illustrates some examples generated by our proposed two poisoning attacks.
Revised: Figure 3 illustrates some examples generated by our proposed two poisoning attacks.

Original: We can see thatthe poisoning samples produced by our clean-label poisoning method look natural to human eyes,which are hard to detect by manual moderation.cat cat horse deer ship cat cat horse deer ship
Revised: We can see that the poisoning samples produced by our clean-label poisoning method look natural to human eyes, which are hard to detect by manual moderation. [Equation] [Figure] ship cat cat horse deer ship

Original:  Evaluation 
Revised:  Evaluation 4.1 Experimental Setup

Original: 4.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: In our experiment, we split each dataset into three portions: the clean training dataset
Revised: In our experiment, we split each dataset into three portions: the clean training dataset

Original: D clean containing the members, the test dataset D test containing the non-members, and the shadowdataset D shadow for generating poisoning samples.
Revised: D clean containing the members, the test dataset D test containing the non-members, and the shadow dataset D shadow for generating poisoning samples.

Original: We follow the same setup with [29], where|D clean | =
Revised: We follow the same setup with [38], where |D clean | =

Original: |D test | =
Revised: |D test | =

Original: |D shadow | , and each of them does not overlap with others.
Revised: |D shadow | , and each of them does not overlap with others.

Original: Additionally, we set thethree datasets to be balanced for the simplicity of evaluation among each class.
Revised: Additionally, we set the three datasets to be balanced for the simplicity of evaluation among each class.

Original: We adopt five datasetsfor our experiments, including (1) MNIST [1] that contains 60,000 handwritten digits from 0 to 9.(2) CIFAR-10 [2] that contains 60,000 images from 10 classes.
Revised: We adopt five datasets for our experiments, including (1) MNIST [1] that contains 60,000 handwritten digits from 0 to 9. (2) CIFAR-10 [2] that contains 60,000 images from 10 classes.

Original: (3) STL-10 [3] that contains 130,000labeled images from 10 classes.
Revised: (3) STL-10 [3] that contains 130, labeled images from 10 classes.

Original: (4) CelebA [17] that contains 202,599 face images annotated by40 attributes.
Revised: (4) CelebA [25] that contains 202,599 face images annotated by 40 attributes.

Original: In our experiment, we train a two-class classifier is to predict the Charming attribute;(5) PatchCamelyon [37] that contains 327,680 images to predict presence of metastatic tissue.
Revised: In our experiment, we train a two-class classifier is to predict the Charming attribute; (5) PatchCamelyon [46] that contains 327,680 images to predict presence of metastatic tissue.

Original: For
Revised: For

Original: MNIST, CIFAR-10, CelebA, and PatchCamelyon dataset, we set |D clean | =10,000, while for STL-10,we set |D clean | =4,000.
Revised: MNIST, CIFAR-10, CelebA, and PatchCamelyon dataset, we set |D clean | =10,000, while for STL-10, we set |D clean | =4,000.

Original: Each sample is transformed into a 96 × 96 × 3 RGB image.
Revised: Each sample is transformed into a 96 × 96 × 3 RGB image.

Original: Models.
Revised: Models.

Original: We use five pretrained models provided by Tensorflow(v2.5.2): Xception, ResNet18,
Revised: We use five pretrained models provided by Tensorflow(v2.5.2): Xception, ResNet18,

Original: MobileNetv2, InceptionV3, and VGG16.
Revised: MobileNetv2, InceptionV3, and VGG16.

Original: For each model, we remove the fully connected (FC) layersto build up the feature extractor g ( · ) .
Revised: For each model, we remove the fully connected (FC) layers to build up the feature extractor g ( · ) .

Original: Then, we add two FC layers on top of the feature extractor toform c ( · ) : one layer with 128 hidden units using the Tanh activation function, followed by an outputlayer using the Softmax activation function.
Revised: Then, we add two FC layers on top of the feature extractor to form c ( · ) : one layer with 128 hidden units using the Tanh activation function, followed by an output layer using the Softmax activation function.

Original: We fix the feature extractor and train the newly added
Revised: We fix the feature extractor and train the newly added

Original: FC layers with the Adam optimizer, with the learning rate of 10 − 5 and batch size of 100.
Revised: FC layers with the Adam optimizer, with the learning rate of 10 − 3 and batch size of 100.

Original: Poisoning Setup.
Revised: Poisoning Setup.

Original: For each learning task, we mount the poisoning attack against each class.
Revised: For each learning task, we mount the poisoning attack against each class.

Original: Weset the poisoning budget b poison as |D clean | # classes .
Revised: We set the poisoning budget b poison as |D clean | # classes .

Original: To make our poisoning attack stealthy to the victim,we assume D poison is evenly distributed among all classes with each having approximately b poison # classespoisoned data samples.
Revised: To make our poisoning attack stealthy to the victim, we assume D poison is evenly distributed among all classes with each having approximately b poison # classes poisoned data samples.

Original: For instance, for a CIFAR-10 classifier, we set b poison = 1 , 000 , where only100 samples for each class are in D poison .
Revised: For instance, for a CIFAR-10 classifier, we set b poison = 1 , 000 , where only 100 samples for each class are in D poison .

Original: Note that this assumption means that there are b poison # classes cleansamples in D clean for the target class t .
Revised: Note that this assumption means that there are b poison # classes clean samples in D clean for the target class t .

Original: For the clean-label poisoning dataset, we set the perturbationconstraint ϵ = 16 / 255 .
Revised: For the clean-label poisoning dataset, we set the perturbation constraint ϵ = 16 / 255 .

Original: Equipment.
Revised: Equipment.

Original: Our experiments were conducted on a deep learning server, which equipped with an
Revised: Our experiments were conducted on a deep learning server, which equipped with an

Original: Intel(R)
Revised: Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Gold 6226R CPU @
Revised: Gold 6226R CPU @

Original: 2.90GHz, 128GB RAM, and four NVIDIA GeForce RTX3090 GPUs with 24GB of memory.
Revised: 2.90GHz, 128GB RAM, and four NVIDIA GeForce RTX 3090 GPUs with 24GB of memory.

Original: Without Poisoning

Original: Dirty-Label Poisoning

Original: [Table] [Table] STLXception . [Table]
Revised: 4.2 Results

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: InceptionV .

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: VGG .

Original: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: Table 1 reports the evaluation results on our proposed two poisoning attacks.

Original: ResNet .

Original: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: In general, we can observe that our attacks obviously increase the AUC score of the membership inference against the target class t , with slight testing accuracy decay.

Original: MobileNetV .

Original: [Table]

Original: [Table]

Original: [Table] [Table] CelebA Xception .
Revised: For instance, for the clean-label poisoning attack against the STL-10 Xception-based classifier, the membership inference AUC is increased from 0.578 to 0.870 on average, while the testing accuracy is decreased from 0.861 to 0.837 on average.

Original: [Table] [Table] [Table] [Table]
Revised: The results show that the poisoned samples resemble the clean samples and the performance of the

Original: [Table]

Original: [Table]

Original: InceptionV .

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: VGG .

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: ResNet .

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: MobileNetV .

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: PatchCamelyon

Original: Xception .

Original: [Table]

Original: [Table]

Original: [Table]We run the attack and evaluation over each class, and we report the average value with standard deviation for each metric. 4.2 Results Table 1 reports the evaluation results on our proposed two poisoning attacks.
Revised: [Table]

Original: In general, we canobserve that our attacks obviously increase the AUC score of the membership inference against thetarget class t , with slight testing accuracy decay. For instance, for the clean-label poisoning attackagainst the STL-10 Xception-based classifier, the membership inference AUC is increased from 0.578to 0.870 on average, while the testing accuracy is decreased from 0.861 to 0.837 on average.
Revised: We run the attack and evaluation over each class, and we report the average value with standard deviation for each metric.

Original: Theresults imply that our clean-poisoning attack is hard to detect: the poisoned samples resemble theclean samples and the performance of the poisoned models degrades subtly on the testing dataset. 
Revised: poisoned models degrades subtly on the testing dataset. Consequently, there are chances that the clean-label poisoning attack evade the manual inspection, when the classification performance for each class does not get carefully examined.

Original: Another observation is that the dirty-label poisoning attack causes more significant membershipexposure than the clean-label poisoning attack.
Revised: Another observation is that the dirty-label poisoning attack causes more significant membership exposure than the clean-label poisoning attack.

Original: A potential explanation of this phenomenon is thatour clean-label poisoning attack can be considered as an “approximate” version of the dirty-labelpoisoning attack in the feature space.
Revised: A potential explanation of this phenomenon is that our clean-label poisoning attack can be considered as an “approximate” version of the dirty-label poisoning attack in the feature space.

Original:  Ablation Study 
Revised:  Ablation Study 5.1 Study 1: Impact of D shadow

Original: 5.1 Study 1: Impact of D shadow

Original: In our prior study, we assume the attacker can obtain a shadow dataset D shadow with the samedistribution of the victim’s clean training dataset D clean .
Revised: In our prior study, we assume the attacker can obtain a shadow dataset D shadow with the same distribution of the victim’s clean training dataset D clean .

Original: However, the model developer may hidethe training dataset information to fortify privacy and intellectual property (IP) protection.
Revised: However, the model developer may hide the training dataset information to fortify privacy and intellectual property (IP) protection.

Original: In thisablation section, we examine how the D shadow affects the attack performance.
Revised: In this ablation section, we examine how the D shadow affects the attack performance.

Original: In our experiment, we assume the attacker aims to poison the cat and aircraft categories against the
Revised: In our experiment, we assume the attacker aims to poison the cat and aircraft categories against the

Original: CIFAR-10 classifiers respectively.
Revised: CIFAR-10 classifiers respectively.

Original: For each target class t , we select 1,000 samples from t in STL-10.
Revised: For each target class t , we select 1,000 samples from t in STL-10.

Original: In this ablation study, We start with the dirty-label poisoning attack to estimate the upper boundperformance of our clean-label poisoning attack.
Revised: In this ablation study, We start with the dirty-label poisoning attack to estimate the upper bound performance of our clean-label poisoning attack.

Original: Table 2 reports the attack performance.
Revised: Table 2 reports the attack performance.

Original: Our resultsshow that the attack performance declines when the attacker’s shadow data has a different distributionfrom the actual clean dataset.[Table]
Revised: Our results show that the attack performance declines when the attacker’s shadow data has a different distribution from the actual clean dataset.

Original: 5.2 Study 2: Impact of b poison
Revised: 5.2 Study 2: Impact of b poison

Original:  [Figure]Figure
Revised: [Table] [Table] [Figure]

Original: We then study how the size of poisoning samplesaffects the attack performance.
Revised: We then study how the size of poisoning samples affects the attack performance.

Original: Figure 4 depicts themembership exposure under dirty-label poisoningattacks with different poisoning budget b poison .
Revised: Figure 4 depicts the membership exposure under dirty-label poisoning attacks with different poisoning budget b poison .

Original: Unsurprisingly, as we gradually decrease the poisoningbudget, the membership inference AUC score decreases as well.
Revised: Unsurprisingly, as we gradually decrease the poisoning budget, the membership inference AUC score decreases as well.

Original: Yet, it is still possible to achieve asignificant membership exposure increase even witha small poison budget.
Revised: Yet, it is still possible to achieve a significant membership exposure increase even with a small poison budget.

Original: Take ResNet50 for instance,when b poison is 100, i.e., the amount of poisoning samples is only 1% of that for the clean samples, we can improve the membership inference AUC scorefrom 0.5971 to 0.7076 on average.
Revised: Take ResNet50 for instance, when b poison is 100, i [Figure] e., the amount of poisoning samples is only 1% of that for the clean samples, we can improve the membership inference AUC score from 0.5971 to 0.7076 on average.

Original: 5.3 Impact of Fine-tuning
Revised: 5.3 Impact of Fine-tuning

Revised: [Figure]

Original:  In the transfer learning scenario, there are also chances that model developers fine-tune parameters offeature extractors [38, 9, 27].
Revised: 468(c) Poisons 02468( [Figure] In the transfer learning scenario, there are also chances that model developers fine-tune parameters of feature extractors [47, 14, 36].

Original: The fine-tuning operation is supposed to make the transfer learningmodel better fit the new dataset.
Revised: The fine-tuning operation is supposed to make the transfer learning model better fit the new dataset.

Original: Inevitably, it may affect how the feature extractor extracts latentfeatures from the inputs.
Revised: Inevitably, it may affect how the feature extractor extracts latent features from the inputs.

Original: As a result, our poisoning attack may get affected.
Revised: As a result, our poisoning attack may get affected.

Original: In this part, we explorehow the fine-tuning process impacts our attack performance.
Revised: In this part, we explore how the fine-tuning process impacts our attack performance.

Original: In our experiment, we train CIFAR-10 classifiers with fine-tuning on the poisoned dataset, with thelearning rate set as 1e-5.
Revised: In our experiment, we train CIFAR-10 classifiers with fine-tuning on the poisoned dataset, with the learning rate set as 1e-5.

Original: We report the attack performance in Table 3.
Revised: We report the attack performance in Table 3.

Original: It can be seen that, for ourdirty-label poisoning attack, we achieve more significant membership exposure amplifications.
Revised: It can be seen that, for our dirty-label poisoning attack, we achieve more significant membership exposure amplifications.

Original: Whilefor our clean-label poisoning attack, the membership exposure changes are nearly negligible.Without Poisoning Dirty-Label Poisoning Clean-
Revised: While for our clean-label poisoning attack, the membership exposure changes are nearly negligible.

Original: [Table]

Original: MobileNetV

Original: [Table]

Original: We report the membership inference (MI) AUC difference and testing accuracy between the poisoned model and the clean model.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: (a)

Original: [Figure]

Original: [Figure]

Original: 2468(b)

Original: Poisons

Original: [Figure]

Original: [Figure]

Original: 468(c)

Original: Poisons

Original: [Figure]

Original: 2468(d) model.

Original: The colored points are the clean training samples, while the dark star marks are the poisoning sample.

Original: To understand this phenomenon, we investigate how the feature extractor behaves in different cases.
Revised: To understand this phenomenon, we investigate how the feature extractor behaves in different cases.

Original: We visualize the latent features of clean samples and poisoning samples by t-SNE technique [36]in Figure 5.
Revised: We visualize the latent features of clean samples and poisoning samples by t-SNE technique [45] in Figure 5.

Original: We can observe that, for the dirty-label attack in the fine-tuning setting, the poisoningsamples distribute closely to the targeted clean samples in the feature space (as shown in Figure 5c.
Revised: We can observe that, for the dirty-label attack in the fine-tuning setting, the poisoning samples distribute closely to the targeted clean samples in the feature space (as shown in Figure 5c.

Original: Besides, for the clean-label attack in the fine-tuning setting, the poisoning samples distribute amongeach class in the feature space (as shown in Figure 5d).
Revised: Besides, for the clean-label attack in the fine-tuning setting, the poisoning samples distribute among each class in the feature space (as shown in Figure 5d).

Original: It indicates that the more poisoning samples inthe target class, the more significant membership exposure results we can get.
Revised: It indicates that the more poisoning samples in the target class, the more significant membership exposure results we can get.

Original: By comparing Figure 5aand Figure 5b, we can also partially illustrate that dirty-label poisoning tends to achieve better attackperformance than the clean-label poisoning.
Revised: By comparing Figure 5a and Figure 5b, we can also partially illustrate that dirty-label poisoning tends to achieve better attack performance than the clean-label poisoning.

Original: This trend is actually consistent with the intuition ofour attack.
Revised: This trend is actually consistent with the intuition of our attack.

Original: In fact, this is the case called “adversarial training” by some adversarial example defenseliterature [35, 26, 40].
Revised: In fact, this is the case called “adversarial training” by some adversarial example defense literature [44, 35, 49].

Original: During adversarial training, model parameters are gradually tuned to eliminatethe effects of input perturbations.
Revised: During adversarial training, model parameters are gradually tuned to eliminate the effects of input perturbations.

Original:  Conclusion
Revised:  Discussion on Potential Countermeasures

Revised: In general, there are two potential ways to defend our poisoning attacks: One is to detect and filter out poisoning data, while the other is to reduce membership exposure. We will investigate the former direction in the future, and in this part, we mainly explore defenses by limiting membership information leakage, including: • Regularization.

Revised: Overfitting is considered to be one of the major culprits of membership exposure [38, 51, 33].

Revised: Therefore, the regularization technique may be feasible to defend against our attacks.

Revised: In our experiment, we introduce an L 2 -norm regularizer with a penalty of 0.05 during the model training process.

Revised: • Early stopping.

Revised: Early stopping is another common practice to prevent overfitting [21].

Revised: During the training process, for each epoch, we randomly sample out 10% of the training data as validation data and use 90% of other data as training data.

Revised: We monitor the validation loss, and if it does not decrease in three epochs, we stop the training process.

Revised: • DP-SGD.

Revised: Differential privacy (DP) provides a rigorous guarantee to limit privacy leakage [12, 13].

Revised: Recently, privacy-preserving machine learning algorithms under the differential privacy have been proposed [41, 6, 18], among which differentially private stochastic descent (DP-SGD) [4] receives the most attention [20, 5, 28].

Revised: In our experiment, we utilize the DP-SGD optimizer provided by the TensorFlow Privacy package 4 to implement differential privacy training.

Revised: The hyperparameters used in our implementation are summarized in Table 4.

Revised: As reported by the analysis tool provided by TensorFlow Privacy, we achieve (3.25, 10 − 6 )-differential privacy on the clean model ( |D train | =

Revised: |D clean | =10,000), while we achieve (3.10, 10 − 6 )-differential privacy on poisoned models ( b poison = 1 , 000 , |D train | =

Revised: |D clean ∪ D poison | =11,000).

Revised: We evaluate the three potential countermeasures on the CIFAR-10 dataset and summarize the results in Table 5.

Revised: By comparing the membership exposure between unprotected (Table 1) and protected models (Table 5), the three defenses can help weaken the membership exposure amplification effect by our attacks.

Revised: Besides, in our experiment, we observe that the regularization technique has the best model utility-privacy trade-off.

Revised: The regularization technique achieves a relatively high model testing accuracy and keeps the AUC scores at lower levels.

Revised: Hyperparameter

Revised: Value

Revised: [Table]

Revised: RATE

Revised: [Table]

Revised: NOISE MULTIPLIER 1.

Revised: MAX L 2 - NORM OF GRADIENTS 1.

Revised: [Table]

Revised: SIZE

Revised: [Table]

Original:  In this paper, we demonstrate the feasibility of exploiting data poisoning to amplify the membershipexposure of the training dataset.
Revised: [Table] [Table]  Conclusion In this paper, we demonstrate the feasibility of exploiting data poisoning to amplify the membership exposure of the training dataset.

Original: We present one significantly increase the precision of the membershipinference attack on classes with limited negative influences on the model’s test-time performance.
Revised: We present one significantly increase the precision of the membership inference attack on classes with limited negative influences on the model’s test-time performance.

Original: We also conduct extensive evaluations to study how different factors may affect attack performance.
Revised: We also conduct extensive evaluations to study how different factors may affect attack performance.

Original: Our findings uncover a new challenge to modern machine learning ecosystems.
Revised: Our findings uncover a new challenge to modern machine learning ecosystems.

Original: Data from the openworld may not only affect the model performance but also raise real threats to private training data.
Revised: Data from the open world may not only affect the model performance but also raise real threats to private training data.

Original: Even worse, we show it is possible to mount clean-label attacks to evade human moderation.
Revised: Even worse, we show it is possible to mount clean-label attacks to evade human moderation.

Revised: Our research is thus a call to action.

Original: Oneprimary limitation of our clean-label attacks is that we assume the attacker has knowledge of thefeature extractor.
Revised: One primary limitation of our clean-label attacks is that we assume the attacker has knowledge of the feature extractor.

Original: How to create clean-label poisons in a black-box manner and how to defend suchprivacy-violating poisoning attacks remain our future work. 
Revised: How to create clean-label poisons in a black-box manner remains our future work. Acknowledgments and Disclosure of Funding We thank the anonymous reviewers for their constructive comments. This work is supported by the

Original: Checklist
Revised: National Key Research and Development Program of China (2020AAA0107702), National Natural Science Foundation of China (U21B2018, 62161160337, 62132011), Shaanxi Province Key Industry Innovation Program (2021ZDLGY01-02), the Research Grants Council of Hong Kong under Grants N_CityU139/21, R6021-20F, R1012-21, C2004-21G

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 6.
Revised: [Yes] See Section 7.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Sec- tion 6.
Revised: [Yes] See Section 7.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments...
Revised: If you ran experiments.

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We will submitthe code, link to data we used, and instructions in the supplemental material.
Revised: [Yes] We will submit the code, link to data we used, and instructions in the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] We reset the random seed (54321) at the beginning ofbuilding clean models and poisoned models.
Revised: [No] We reset the random seed (54321) at the beginning of building clean models and poisoned models.

Original: (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 4.1.
Revised: [Yes] See Section 4.1.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We cite the data and models we used in Section 4.1.
Revised: [Yes] We cite the data and models we used in Section 4.1.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] We did not use any new assets.
Revised: [No] We did not use any new assets.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [No] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [No] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No]
Revised: [No]

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]