Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning
Authors: Harsh Chaudhari
Giorgio Severi
Alina Oprea
Jonathan Ullman
Venue: ICLR 2024 poster (2024)
Keywords: Privacy Attack
Membership Inference
Data Poisoning
Primary area: societal considerations including fairness, safety, privacy

Abstract: The integration of Machine Learning (ML) in numerous critical applications introduces a range of privacy concerns for individuals who provide their datasets for ML training purposes. One such privacy risk is Membership Inference (MI), in which an adversary seeks to determine whether a particular data point was included in the training dataset of a model. Current state-of-the-art MI approaches capitalize on access to the model’s predicted confidence scores to successfully perform membership inference, and employ data poisoning to further enhance their effectiveness. 
In this work, we  focus on the less explored and more realistic label-only setting, where the model provides only the predicted label as output. We show that existing label-only attacks are ineffective at inferring membership in the low False Positive Rate (FPR) regime. To address this challenge,  we propose a new attack Chameleon that leverages a novel data poisoning strategy and an efficient query selection method to achieve significantly more accurate membership inference than existing label-only attacks, especially for low FPRs.

Published as a conference paper at ICLR 2024
CHAMELEON: INCREASING LABEL-ONLY MEMBER- SHIP LEAKAGE WITH ADAPTIVE POISONING
Harsh Chaudhari, Giorgio Severi, Alina Oprea, Jonathan Ullman Khoury College of Computer Sciences
Northeastern University Boston, MA 02115, USA
(chaudhari.. .ha, severi.g, opreaa, j.ullman) @northeastern.en /noestennoottteesseemmmmmmmm
ABSTRACT
The integration of machine learning (MI.) in numerous critical applications intro- duces a range of privacy concerns for individuals who provide their datasets for model training. One such privacy risk is Membership Inference (MI), in which an attacker seeks to determine whether particular data sample was included in the training dataset of model. Current state-of-the-as MI attacks capitalize on access the model's predicted confidence scores to successfully perform membership inference, and employ data poisoning to further enhance their effectiveness. In this work, focus the less explored and more realistic label-only setting, where the model provides only the predicted label on a queried sample. We show that existing label-only MI attacks are ineffective inferring membership in the low False Positive Rate (FPR) regime. To address this challenge, we propose a new attack Chameleon that leverages a novel adaptive data poisoning strategy and an efficient query selection method to achieve significantly more accurate membership inference than existing label-only attacks, especially at low FPRs.
INTRODUCTION
The use of machine learning for training on confidential or sensitive data, such as medical records (Stanfill et al.) (010). financial documents (Ngai et al. 2011). and conversations (Carlini et al., 2021 introduces range of privacy violations. By interacting with trained MI. model, an attacker might reconstruct data from the training set (Haim et al.) 2022 Balle et al. 2022), perform membership inference (Shokri et al. 2017; Yeom et al. 20188) Carlini et al. 2022) or learn sensitive attributes from training data (Fredrikson et al. 2015 Mehnaz al. (022). Membership inference (MI) attacks (Shokri et al. 2017), originally introduced under the name of tracing attacks (Homer et al. 2008). enable attacker to determine whether not data sample was included in the training set of an ML model. While these attacks are less severe than training data reconstruction, they might still constitute serious privacy violation. Consider mental health clinic that uses MI. model to predict patient treatment responses based on medical histories. An attacker with accesses to certain individual's medical history can learn if the individual has mental health condition by performing successful MI attack.
We can categorize MI attacks into two groups: confidence-based attacks in which the attacker gets access to the target ML model's predicted confidences, and label-only attacks, in which the attacker obtains only the predicted label for queried samples. Recent literature has primarily focused on confidence-bant based attacks Carlini et al. (2022); [errtan et al. (2023) that maximize the attacker's success at low Palse-Position Rates (FPRs). Additionally, Tramer et al. (2022) and et al. (2022) showed that introducing data poisoning during training significantly improves the MI performance low FPRs in the confidence-based scenario.
Nevertheless, in many real-world scenarios, organizations training ML models respond only with hard labels to customer queries. For example, financial institutions might solely indicate whether a customer has been granted home loan credit card approval. In such cases, launching an MI attack gets considerably more challenging as the attacker looses access to prediction confidences and cannot leverage state-of-the-art attacks such Carlini al. (2022). Wen et al. (2023). (Berrran al. (003))

---

Published as a conference paper at ICLR 2024
Furthermore, it remains unclear whether existing label-only MI attacks, such as Choquette- -hhoo et al. (2021) and Li & Zhang (202I), are effective in the low FPR regime and data poisoning techniques can be used to amplity the membership leakage in this specific realistic scenario.
In this paper. we first show that existing label-only MI attacks (Yom et al. 2018 Choquette- Choo et al.] [021] Li & Zhang, 2021) struggle to achieve high True Positive Rate (TPR) in the low HPR regime. We then demonstrate that integrating state-of-the-an data poisoning technique (Tramer et al.)
2022) into these label-only MI attacks further degrades their performance, resulting in even lower TPR values at the same FPR. We investigate the source of this failure and propose a new label-only MI attack Chameleon that leverages a novel adaptive poisoning strategy to enhance membership inference leakage in the label-only setting. Our attack also uses an efficient querying strategy. which requires only 64 queries to the target model succeed in the distinguishing test, unlike prior works (Choquette- Chho et al. 2021 Li & Zhhngg [021) that use on the order of few thousand queries. Extensive experimentation across multiple datasets shows that our Chameleon attack consistently outperforms previous label-only MI attacks, with improvements in TPR at I 1% FPR ranging up to 17.5x. Finally, we also provide a theoretical analysis that sheds light on how data poisoning amplifies membership leakage in label-only scenarios. To the best of our knowledge, this work represents the first analysis on understanding the impact of poisoning on MI attacks.
BACKGROUND AND THREAT MODEL
We provide background on membership inference, describe our label-only threat model with poison- ing, and analyze existing approaches to motivate our new attack.
Related Work. Membership Inference attacks can be characterized into different types based on the level of adversarial knowledge required for the attack. Full-knowledge (or white-box) attacks (Nasr et al., 2018 Leino& & Fredrikson 2020) assume the adversary has access to the internal weights of the model, and therefore the activation values of each layer. In black-box settings the adversary can only query the ML model, for instance through an API, which may return either confidence scores or hard labels. The confidence setting has been studied most, with works like Shokri et al. (2017);
Carlini et al. 2022]; Ye et al. (2022) training multiple shadow models local surrogate models- and modeling the loss (or logit) distributions for members and non-memeers
The label-only MI setting, investigated by Yeom et al. (2018); Choquette- Choo et al. (2021) Li
Zhang 2001)) considers more realistic threat model that returns only the predicted label on queried sample. Designing MI attacks under this threat model more challenging, as the attack cannot rely on separating the model's confidence on members and non-members. Existing label-only MI attacks are based on analyzing the effects of perturbations on the original point on the model's decision. With our work we aim to improve the understanding of MI in the label-only setting, especially in light of recent trends in MI literature.
Current MI research, in fact, is shifting the attention towards attacks that achieve high True Positive Rates (TPR) in low False Positive Rates (FPR) regimes (Carlini et al. 2022: Ye et al., 2022 Luu et al. 2022, Wen et al. 2023 Bertran al. 2023). These recent papers argue that if an attack can manage to reliably breach the privacy of even a small number of, potentially vulnerable, users, it is still extremely relevant, despite resulting in potentially lower average-case success rates. A second influential research thread exposed the effect that training data poisoning has on amplifying privacy risks. This threat model is particularly relevant when the training data crowd-sourced, or obtained through automated crawling (common for large datasets), as well as in collaborative learning settings. Tramer al. (002)) and Chen et al. (202)) showed that data poisoning amplifies MI privacy leakage and increases the TPR values at low FPRs. Both LiRA (Carlini et al.) 2022) and Truth Serum (Tramer jt al. 2022) use large number of shadow models (typically 128) to learn the distribution of model confidences, but these methods do not directly apply to label-only membership inference, much more challenging setting. A related line of research (Mahloujifar et al. 2022 Chaudhari et al. 2023) showcased how data poisoning could be utilized to infer statistical information about the properties of the training set, called property inference attacks.
Threat Model. We follow the threat model of Tramer et al. (202 used for membership inference with data poisoning, with adjustments to account for the more realistic label-only setting. The attacker has black-box query access the ability send samples and obtain the corresponding outputs- to
2

---

Published as a conference paper at ICLR 2024
CCNMS OUT COWPS
COMPS OUT CONPS
COM'S
GUI CORPS
Challenge Paint Potson)
Challange Point D Potsen) IN cOMPS OUT coNYs
Challange Point B Poniona IN COMS
CoMP OUT cowPS
OUT -OWP5
Figure 1: Distribution of confidence scores for two challenge points (a Car and a Ship), highlighting the impact of poisoning on two different points of the CIFAR-10 1 dataset, Each row shows the shift in model confidences (wrt. true label) for IN and OUT models with introduction of poisoned samples.
trained machine learning model M,, also called target model, that returns only the predicted label on an input query. The attacker's objective is to determine whether a particular challenge poin! was part of M, training set or not. Similarly to Tramèr al. 2022) the attacker has the capability to inject additional poisoned data Dp into the training data Drr sampled from a data distribution D. The attacker can only inject Dp once before the training process begins, and the adversary does not participate further in the training process after injecting the poisoned samples. After training completes, the adversary can only interact with the final trained model to obtain predicted labels on selected queried samples. Following the MI literature, A can also train local shadow models with data from the same distribution as the target model's ' training set. Shadow models training sets may may not include the challenge points. We will call IN models those trained with the challenge point included in the training set, and OUT models those trained without the challenge point.
Analyzing Existing Approaches. Existing label-only MI attacks (Choquette- Cheuu al. 202 & Zhang 2021) ) propose a decision boundary technique that exploits the existence of adversarial examples to create their distinguishing test. These approaches typically require a large number of queries to the target model to estimate sample's is distance to the model decision boundary. However, these attacks achieve low TPR (e-g., 1.1% at 1% FPR when tested the CIFAR-10 dataset. In contrast, the LiRA confidence- baaddd attack by Carlimi et al. (2022) achieves a TPR of 16.2% at 1% FPR on the same dataset. Truth Serum (Tramer et al.) (202) evaluates LiRA with data poisoning strategy based on label flipping, which significantly increases TPR to 91.4% at 1 % FPR once 8 poisoned samples are inserted per challenge point.
A natural first strategy for label-only MI with poisoning is to incorporate the Truth Serum data poisoning method the existing label-only MI attack (Choquette- -Choo al.) (02T]) and investigate if the TPR at low FPR can be improved. The Truth Serum poisoning strategy is simply label flipping, where poisoned samples have identical features to the challenge point, but different label. Surprisingly, the results show negative outcome, with the TPR decreasing to 0% at 1% FPR after poisoning. This setback compels us to reconsider the role of data poisoning in improving privacy attacks within the label-only MI threat model. We question whether data poisoning improve label-only MI, and if so, why did our attempt to combine the two approaches fail. In the following section, we provide comprehensive answers these questions and present novel poisoning strategy that significantly improves the attack success in the label-only MI setting.
indeed
3
CHAMELEON ATTACK
We first provide some key insights for our attack, then describe the detailed attack procedure, and include some analysis on leakage under MI.
3.1 ATTACK INTUITION
Given the threat model, goal is improve the TPR in the low FPR regime for label-only MI, while reducing the number of queries to the target model M,. To achieve this two-fold objective, we start by addressing the fundamental question of determining an effective poisoning strategy. This
3

---

Published as a conference paper at ICLR 2024
Algorithm 1 Adaptive Poisoning Strategy
Input: Challenge point 1)) poisoned point (ær y²) where y' # y, attacker's dataset Dadi, poison threshold tp and maximum poisoned iterations Kmax
I: Let k denote the number of poisoned replicas.
2: For k == O.... Kmax do:
3: Construct poisoned dataset D, containing & replicas of (x. (x).y).
4: Train OUT models {0Tmh ...... ..... ( ) dataset D U Davv
5:
Query x on OUT models and obtain confidences cf.--., c for label y, where 0 S o <
6: Compute mean of the confidences μ E.
7: If S lp: break
8:
9: ==
Output: Number poisoned replicas A.
involves striking the right balance such that the IN models classify the point correctly, and the OUT models misclassify it. Without poisoning, it is likely that both IN and OUT models will classify the point correctly (up to some small training and generalization error). Conversely, if we insert too many poisoned samples with an incorrect label, then both IN and OUT models will misclassify the point to the incorrect label. As the attacker only gets access to the labels of the queried samples from Mr, over-poisoning would make it implausible to distinguish whether the model is an IN or OUT model.
The state-of-tue of------tthttte art Truth Serum attack (Tramer et al., 2022), which requires access to model confidences, employs static poisoning strategy by adding fixed set of k poisoned replicas for each challenge point. This poisoning strategy fails for label-only MI as often times both IN and OUT models misclassify the target sample, as discussed in Section 2 Our crucial observation is that not all challenge points require the same number of poisoned replicas to create a separation between IN and OUT models. To provide evidence for this insight, we show visual illustration of model confidences under the same number of poisoned replicas for two challenge points in Figure Therefore, we propose
strategy that adaptively selects the number of poisoned replicas for each challenge point, with the goal of creating separation between IN and OUT models. The IN models trained with the challenge point in the training set should classify the point correctly, while the OUT models should misclassify it. Our strategy adaptively adds poisoned replicas until the OUT models consistently misclassify the challenge point a significantly higher rate than the IN models.
Existing MI attacks with poisoning (Tramer et al., 2022 Chen et al. (222)) utilize confidence scores obtained from the trained model to build a distinguishing test. As our attacker only obtains the predicted labels, we develop a label-only "proxy" metric for estimating the model's 's confidence, by leveraging the predictions obtained on "close" neighbors of the challenge point. We introduce the concept of a membership neighborhood, which is constructed by selecting the closest neighbors based on the KL divergence computed on model confidences. This systematic selection helps us improve the effectiveness of our attack by strategically incorporating only the relevant neighbor predictions. The final component of the attack is the distinguishing test, which we compute a score based on the membership neighborhood set. These scores
the target model M, 's correct predictions used to compute the TPR at fixed FPR values, as well as other metrics of interest such as AUC and MI accuracy. We provide detailed description for each stage of our attack below.
3.2 ATTACK DETAILS
IN Modes
vosel conteonco on rue Label
GUI Models
Our Chameleon attack can be described as a three-stage process:
Adaptive Poisoning. Given challenge point (x,y) and access to the underlying training distribution D, the attacker constructs a training dataset Dadv D. such that (x, y) & Dad and poisoned replica (x. Y'), for some label y * y. The goal of the attacker is to construct small enough poisoned set Dp such that model trained on Dadv U Dp. which excludes (z, y), missclassifies the challenge point. The attacker needs to train their own OUT shadow models
Mimber porstried repi
Figure 2: Impact of poisoning on confidences of IN and OUT models (wrt. true label) for point in CIFAR- 10 dataset.
4

---

Published as a conference paper at ICLR 2024
Chalange Poin Peleed delf strels
Cisso Siighte hiland di
Rorrali Nelehbe 14 Pnllant CITY Oif
Netplter
M-ltogy 2a09t
-50
Figure 3: Effect of poisoning on the scaled confidence (logit) distribution of challenge point and its neighbors. Both IN and OUT distributions of near neighbor, unlike the far-away neighbor, exhibit behavior similar to the challenge point distribution before and after introduction poisoning.
(without the challenge point) to determine how many poisoned replicas are enough to mis-classify the challenge point. Using that any other OUT model (e.g., the target model M,) has a similar behavior under poisoning. The attacker begins with no poisoned replicas and trains m shadow models on the training set Dadv- The attacker adds poisoned replica if the average confidence across the OUT models on label above a threshold tp. and repeats the process until the models' average confidence on label y falls below tp (threshold where mis classiassifiittton for the challenge point occurs). The details of our adaptive poisoning strategy are outlined in Algorithm [ which describes the iterative procedure for constructing the poisoned set.
of OUT models instead of single one increases the chance
Note that we do not need to separately train any IN models, i.e., models trained on DDUDadv ((xr, v)}. to select the number of poisoned replicas for our challenge point. This is due our observation that, in presence of poisoning, the average confidence for the true label tends to be higher on the IN models when compared to the OUT models. Figure 2 jillustrates where the average confidence on the OUT models decreases at a faster rate than the confidence on the IN models with the addition of more poisoned replicas. As a result, the confidence mean computed on the OUT models (line 6 in Algorithm D will always cross the poisoning threshold tp first, leading misclassification of the challenge point by the OUT models before the IN models. Therefore, we are only required to train OUT models for our adaptive poisoning strategy.
instance of this phenomenon,
Multiple Challenge Points: In practical scenarios, an attacker aims to infer membership across multiple challenge points rather than focusing on a single point. In Appendix C we propose a strategy that handles set of challenge points simultaneously while naturally capturing interactions among them during the poisoning phase. Importantly, our strategy only incurs fixed overhead cost, enabling the attack to scale to any number of challenge points. For resource-limited scenarios, we also propose an optimized variant of our attack requiring just 16 shadow models while outperforming prior label-only attacks. A detailed description and cost analysis is reported in Appendix C a D
Membership Neighborhood. In this stage, the attacker's objective is to create membership neighborhood set ((2.9) by selecting close neighboring points to the challenge point. This set is then used to compute a proxy score in order to build distinguishing test. To construct the neighborhood set, the attacker needs N shadow models such that the challenge point (ar, y) appears the training set of half of them (IN models), and not in the other half (OUT models). Interestingly, the attacker can reuse the OUT models trained from the previous stage and reduce the computational cost of the process. Using these shadow models, the attacker constructs the neighhorhood set 5(z.9) for given challenge point (x. v). A candidate (Ie, y). where &e + x, is said to be in set s(r..)) if the original point and the candidate's model confidences close in terms of KL divergence for both IN and OUT models, i.e., the following conditions are satisfied:
(1) Here, KL() calculates the Kullback-Leible divergence between two distributions. Notations ((x:))N and (x:))oUT represent the distribution of confidences (wrt. label y) for candidate (I y) on the IN and OUT models trained with respect to challenge point (x, y).
KL & (Ir.)N || ((z))n) ) S tab and KL (=r)oou || (<))our) S tab
5

---

Published as a conference paper at ICLR 2024
Note that the models used in this stage do not need to include poisoning into their training data. We observe that the distribution of confidence values for candidates characterized by low KL divergence tend to undergo similar changes those of the challenge point when poisoning is introduced. We call such candidates close neighbors. Figure 3 we show how the confidence distribution of close neighbor closely mimics the confidence distribution of the challenge point as we add replicas, Additionally, we also demonstrate that the confidence distribution of a remote neighbor hardly affected by the addition of poisoned replicas and does not exhibit a similar shift in its confidence distribution as the challenge point. Therefore, it is enough to train shadow models without poisoning, which reduces the time complexity of the attack.
poisoned
In practical implementation, we approximate the distributions ((xx)) JIN and 4b (xte)oor using a scaled version of confidences known as logits. Previous work (Carlini al.] 2022 Tramer et al. 2022) showed that logits exhibit Gaussian distribution, and therefore we compute the KI. divergence
between the challenge point and the candidate confidences using Gaussians. In Section empirically show the importance of selecting close neighbors.
we
Distinguishing Test. The final goal of the attacker to perform the distinguishing test. Towards this objective, the attacker queries the black-box trained model M using the challenge point and its neighborhood set ((2.,) consisting of close neighbors. The attacker obtains a set of predicted nbb in return and computes the missclassification score of the trained model
labels {ūi..... Mn+1 }
The score fz) denotes the fraction of neighbors whose predicted labels do not match the ground truth label. This score is then used to predict if the challenge point was a part of the training set or not. Correspondingly, we use the computed misclassification score fayy y to calculate various metrics, including TPR @ fixed FPR, AUC, and MI accuracy.
f()), = Eist ルœ
n+1
3.3 LABEL-ONLLY MI ANALYSIS
We now analyze the impact of poisoning on MI leakage in the lahel-only setting. We construct an optimal attack that maximizes the True Positive Rate (TPR) at fixed FPR of 2%, when & poisoned replicas related to a challenge point introduced in the training set. The formulation of this optimal attack is based on a list of assumptions outlined in Appendix B The objective of constructing this optimal attack is twofold. First, we aim to examine how increasing the number of poisoned replicas influences the maximum TPR (@ (@x%FPR) Second, we seek to evaluate whether the behavior of Chameleon aligns with (or diverges from) the behavior exhibited by the optimal attack. In Figure 4 we present both attacks at FPR of 5% on CIFAR- The plot depicting the optimal attack shows an initial increase in the maximum attainable TPR following the introduction of poisoning. However, as the number of poisoned replicas increases, the TPR decreases, indicating that excessive poisoning in the label-only scenario adversely impacts the attack TPR. Notably, Chameleon exhibits a comparable trend to the optimal attack, showing first an increase and successively a decline in TPR poisoning increases. This alignment suggests that our attack closely mimics the behavior of the optimal attack and has similar decline in TPR due to overpoisoning. The details of our underlying assumptions and the optimal attack are given in Appendix B
Optimal Attack Our Attack
30
TP
Number Poisoned Rep liciica (k)
Figure 4:
Comparing Optimal and Our attack under poisoning.
4 EXPERIMENTS
We show that Chameleon significantly improves upon prior label-only MI, then perform several ablation studies, and finally we evaluate if differential privacy (DP) is an effective mitigation.
4.1 EXPERIMENTAL SETTING
We perform experiments on four different datasets: three computer vision datasets (GTSRB, CIFAR- 10 and CIFAR-100) and one tabular dataset (archase-100), We ResNet-18 convolutional neural

---

Published as a conference paper at ICLR 2024
network model for the vision datasets. We follow the standard training procedure used in prior works (Carlimi et al., 2022] Tramer et al. [022] Wen et al. (2023), including weight decay and common data augmentations for image datasets, such random image flips and crops. Each model is trained for 100 epochs, and its training set is constructed by randomly selecting 50% of the original training set.
To instantiate our attack, we pick 500 challenge points random from the original training set. In the adaptive poisoning stage, we set the poisoning threshold E = 0.15, the number of OUT models m 8 and the number of maximum poisoning iterations Kmax - 6. In the membership neighborhood stage, we set the neighborhood threshold tn = 0.75, and the size of the neighborhood | |= 64 samples. Later in Section 4.3] we vary these parameters and explain the rationale behind selecting these values. To construct neighbors in the membership neighborhood, we generate a set of random augmentations for images and select subset of 64 augmentations that satisfy Eqn. [ Finally we test our attack on 64 target models, trained using the same procedure, including the poisoned set. Among these, 32 serve as IN models, and the remainder as OUT models, in relation to each challenge point. Therefore, the evaluation metrics used for comparison are computed over 32,000 observations.
Evaluation Metrics. Consistent with prior work (Carlini et al.) 2022 Chen al. 2022 Tramer et al.) 2022 Wen et al. 2023), our evaluation primarily focuses on True Positive Rate (T (T)) at various False Positive Rates (FPRs) namely 0.1 %2 1%, 5% and 10%. To provide a comprehensive analysis, we also include the AUC (Area Under the Curve) score of the ROC (Receiver Operating Characteristic) curve and the Membership Inference (MI) accuracy when comparing our attack with prior label-only
attacks (Yeom et al. 2018 Choquette-Choo et al. 2021 Li & Zhang 2021.
4.2 CHAMELEON ATTACK IMPROVES LABEL- ONLY MI
We compare our attack against two prior label-only attacks: the Gap attack (Yeom et al. 2018), which predicts any misclassified data point as a non-member and the state-of-the-art
Boundary attack (Choquette- Choquette-Choo et al.] 2021 Li & Zhang (021). which uses a sample's 's distance from the decision boundary to determine its membership status. The Decision- Decision-Boundary attack relies on black-box adversarial example attacks (Brendel et al. 20188; Chen et al. 2020). Given a challenge point (z.yy). the attack starts from random point x for which the model prediction is not label y and walks along the boundary while minimizing the distance to xr. The perturbation needed to create the adversarial example estimates the distance to the decision boundary, and a sample considered to be in the training set if the estimated distance is above a threshold, and outside the training set otherwise. Choquette- Choo et al. (202T) showed that their process closely approximates results obtained with a stronger white-box adversarial example technique (Carlini & Wagner (2017) using z 2,500 queries per challenge point. Consequently, we directly compare with the stronger white-box version and show that our attack outperforms even this upper bound.
Table 1: Comparing Label-only attacks on GTSRB (G-43), CIFAR- (C-10) and CIFAR- 100 (C-100) datasets. Our attack achieves high TPR across various FPR values compared to prior attacks.
TPR@0.1% FFR
IPR@1
TPR05*FPR
TPRØ 10%
Label-Only Atlack
G-43
C-10
C-100
G-43
C-10
C-100
G-43
C-10
C-100
G-43
C-10
C-100
Gap
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
Docision-Boundary
0.04%
0.08%
0.02%
1.1%
1.3%
3.6%
5.4%
5.6%
23.0%
10.4%
11.6%
44.9%
Chameleon (Ours)
3.1%
8.3%
29.6%
11.4%
22.8%
52.5%
25.9%
34.7%
70.9%
35.0%
42.8%
79.4%
Table U provides detailed comparison of Chameleon with prior label-only MI attacks. Chameleon shows a significant TPR improvement over all FPRs compared to prior works. In particular, for the case of TPR 0.1% FPR, prior works achieve TPR values below 0.08%, but Chameleon achieves TPR values ranging from 3. 1% to 29.6% across the three datasets, marking substantial improvement ranging from 77.5x to 370x. At 1% FPR, the TPR improves by a factor between 10.36x and 17.53 x. Additionally, our attack consistently surpasses prior methods in terms of AUC and MI accuracy metrics. A detailed comparison can be found in Table 2 (Appendix I) Notably, Chameleon is significantly more query-efficient, using only 64 queries to the target model, compared to the decision boundarr attack, which requires x 2,500 queries for the MI test, making attack approximately 39x more query-efficient.
Furthermore, Chameleon requires adding a relatively low number of poisoned points per challenge point: an average of 3.5 for GTSRB, 1.4 for CIFAR-10, and 0.6 for C10AR-100 datasets for each
7

---

Published as a conference paper at ICLR 2024
TRIGTSPRE DRIgESFFE EIROUEMIP
Thue Posrivve [TPR)
Tue Pesitvv
Reatn 10C 0.50
2 Novmn sUCeo.57
04
2.2
Palse
Foisoning Threshold
(a) Comparing adaptive and static poisoning.
(h) TPR/AUC by poisoning threshald tp.
Thes Prsitive (TPR)
Paxiive
DuT todelt AUCH0.75
DRTT Modes (1KC-0.0
10"1
False Pasttive
Iterations
(c) TPR/AUC by number of OUT models. (d) TPR/AUC by number iterations Kmax-
Figure 5: Ablations for Adaptive Poisoning stage on CIFAR-10 dataset. We provide experiments by varying vaious hyperparameters used in the Adaptive Poisoning stage.
challenge point. This results in minor drop test accuracy of less than 2%, highlighting the stealthiness of our attack. Overall, the results presented in Table [ show that our adaptive data poisoning strategy significantly amplifies the MI leakage the label-only scenario while having a marginal effect on the model's 's test accuracy.
4.3 ABLATION STUDIES
Adaptive Poisoning Stage. We evaluate the effectiveness of adaptive poisoning and the impact of several parameters.
i) Comparison to Static Poisoning. Figure Sa provides a comparison our adaptive poisoning approach (Algorithm I and static approach where k replicas added per challenge point. Our approach achieves a TPR @ 1% FPR of 22.9%, while the best static approach among the six versions achieve TPR @ 1% FPR of 8.1%. The performance improvement of 14.8% in this metric demonstrates the effectiveness of our adaptive poisoning strategy over static poisoning, strategy used in Truth Serum for confidence-based MI. Additionally, our approach matches the best static approach (with poison) for the AUC metric, achieving an AUC of 76%.
ti) Poisoning Threshold. Figure 5b illustrates the impact of varying the poisoning threshold tp (line of Algorithm|| I on the attack's 's performance. When tp 1, the OUT model's 's confidence is at most 1 and Algorithm will not add any poisoned replicas in the training data, but setting tp < 1 allows the algorithm to introduce poisoning. We observe an immediate improvement in the AUC, indicating the benefits of poisoning over the no-poisoning scenario. Similarly, the TPR improves when tp decreases, as this forces the OUT model's confidence on the true label to be low, increasing the probability of missclassificatio. However, setting tp close to 0 leads to overpoisoning, where an overly restrictive t, forces the algorithm add a large number of poisoned replicas in the training data, negatively impacting the attack's 's performance. Therefore, setting between 0.1 and 0.25 results in high TPR.
tii) Number of OUT Models. Figure Sc shows the impact of the number of OUT models (line 5 of Algorithm I on our attack's 's performance. As we increase the number of OUT models from to 8, the TPR @ 1 % FPR shows noticeable improvement from 13. % 22.9%. However, further increasing
8

---

Published as a conference paper at ICLR 2024
the number of OUT models up to 64 only yields marginal improvement with TPR increasing to 24.1%. Therefore, setting the number of OUT models to 8 strikes a balance between the attack's s success and the computational overhead of training these models.
iv) Maximum Poisoning Iterations. In Figure 5d] we observe that increasing the maximum number of poisoning iterations kmax (line 2 of Algorithm!] I leads to significant improvements in both the TPR and AUC metrics. When kmax = O, it corresponds to the no poisoning case. The TPR @ 1% FPR and AUC metries improve as parameter kmax increases. However, the TPR stabilizes when kmax 2 4, indicating that no more than 4 poisoned replicas are required per challenge point.
Next, we explore the impact of varying the membership neighborhood size and the neighborhood threshold bbb individually. For the neighborhood size, we observe a consistent increase in TPR @ 1 % FPR of 0.6% as we increase the number of queries from 16 to 64, beyond which the TPR oscillates. Thus, we set the neighborhood size at 64 queries for our experiments, achieving satisfactory attack success. For the neighborhood threshold parameter Inb, note decrease in TPR @ 1% FPR of 6.2% as we increase Lnb from 0.25 to 1.75. This aligns with our intuition, that setting tnb smaller value prompts our algorithm to select close neighbors. which in turn enhances our attack's ' performance. The details for these results are in Appendix (..2
Membership Neighborhood Stage.
4.4 OTHER DATA MODALITIES AND ARCHITECTURES
To show the generality of our attack, we evaluate Chameleon on various model architectures, including ResNet-34, ResNet-50 and V6G-16. We observe a similar trend of high TPR value at various FPRs. Particularly for VGG-16, which has 12x more trainable parameters than ResNet-18, the attack achieves better performance than ResNet-18 across all metrics, suggesting that more complex models tend to be more susceptible to privacy leakage. We also evaluate Chameleon against a tabular dataset (Purchase-100), Once again, we find similar pattern of consistently high TPR values across diverse FPR thresholds. In fact, the TPR@ @ 1% FPR metric reaches an impressive 45.8% when tested on a two-layered neural network. Experimental setup details and results can be found in Appendix A.3
4.5 DOES DIFFERENTIAL PRIVACY MITIGATE CHAMELEON ?
Here, we evaluate Chameleon against models trained using a standard differentially private (DP) training algorithm, DP-SGD (Abadi al. 2016). Our evaluation covers broad spectrum of privacy parameters, but here we highlight results on foo, 100, 4), which represent no privacy, a loose level of privacy, and a strict level of privacy, respectively. At e as high as 100, we observe decline in Chameleon's 's performance, with TPR@ @ 1% FPR decreasing from 22.6% (at e bc) to 6.1% In the case of an even stricter ineffective. However, it is also important to note that the model's accuracy also degrades significantly. plummeting from 84.3% accuracy. This trade-off shows that while DP serves as powerful defense, it does come at the expense of model utility. More results using a wider range of values can be found in Appendix(A.A A.4
4, we observe that TPR 1% FPR becomes 0%, making our attack
oc to 49.4% at c = 4, causing a substantial 34.9% decrease in
DISCUSSION AND CONCLUSION
In this work we propose new attack that successfully amplifies Membership Inference leakage in the Label-Only setting. Our attack leverages a novel adaptive poisoning and querying strategy, surpassing the effectiveness of prior label-only attacks. Furthermore, we investigate the viability of Differential Privacy as a defense against our attack, considering its impact on model utility. Finally, we offer theoretical analysis providing insights on the impact of data poisoning on MI leakage. We demonstrated that Chameleon achieves impressive performance in our experiments, mainly due to the adaptive poisoning strategy we design. While poisoning is a core component of our approach, and allows us to enhance the privacy leakage, it also imposes additional burden on the adversary to mount the poisoning attack. One important remaining open problem in label-only MI attacks is how to operate effectively in the low False Positive Rate (FPR) scenario without the assistance of poisoning. Additionally, our poisoning strategy requires training shadow models. Though our approach generally involves low number of shadow models, any training operation is inherently expensive and adds computational complexity. An interesting direction for future work is the design of poisoning strategies for label-only membership inference that do not require shadow model training.
9

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
Our work introduces a new membership inference methodology. that could in principle be used to exacerbate the privacy risks to individuals' private data when included in machine learning training sets. The main reason motivating us to investigate this type of attacks is to shed light on the extent of the risks, and ensure practitioners are aware of them. In comparison with previous works investigating label-only piisnnig-eenaanedd membership inference attacks, we show that the adversary can manage to reliably compromise the privacy of a significant fraction of the training data. We argue that this type of privacy risk should be seriously taken into account and encourage practitioners to adopt private training procedures, such as differentially private training, to minimize it.
ACKNOWLEDGEMENTS
We thank Sushant Agarwal and John Abascal for helpful discussions. Alina Oprea was supported by NSF awards CNS-2120603 and ZNNS-224748. Jonathan Ullman was supported by NSF awards CNS-2120603 CNS-2233692, and CNS-2247484
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016.
Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adversaries. In IEEE Symposium on Security and Privacy (SP), 2022.
Martin Bertran, Shuai Tang, Michael Kearns, Jamie Morgenstemn Aaron Roth, and Zhiwei Steven Wu. Scalable membership inference attacks via quantile regression. In Advances in Neural Information Processing Systems, 2023.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- based adversarial attacks: Reliable attacks against black-box machine learning models. In The Sixth International Conference on Learning Representations, 2018
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), 2017.
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), 2021.
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In IEEE Symposium on Security and Privacy (SP), 2022.
Harsh Chaudhari, John Abascal, Alina Oprea, Matthew Jagielski, Florian Tramèr, and Jonathan Ullman. Snap: Efficient extraction of private properties with poisoning. In 2023 IEEE Symposium on Security and Privacy (SP), pp. 400-417, 2023. doi: 10.1 0009:94621552023.1177777
Jianbo Chen, Michael I. Jordan, and Martin J. Wainwright. Hopskipjumpattack: A query-efficient decision-bbsed attack. 2020 IEEE Symposium on Security and Privacy (SP), 2020.
Yufei Chen, Chao Shen, Yun Shen, Cong Wang, and Yang Zhang. Amplifying membership exposure via data poisoning. Advances in Neural Information Processing Systems, 2022.
Christopher A. Pnoouuette-Choo Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In Proceedings of the 38th International Conference on Machine Learning, 2021.
10

---

Published as a conference paper at ICLR 2024
Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, and Borja Balle. Unlocking high- accuracy differentially private image classification through scale, 2022.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery, 2015.
Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. In Advances in Neural Information Processing Systems, 2022.
Nils Homer, Szaboles Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays. PLoS genetics, 4(8):e 1000167, 2008
Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy. 2022.
Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated [White-Box] membership inference. In 29th USENIX security symposium (USENIX Security 20), pp. 1605-1622, 2020.
Zheng Li and Yang Zhang. Membership leakage in label-only exposures. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021.
Yiyong Liu, Zhengyu Zhao, Michael Backes, and Yang Zhang. Membership inference attacks by exploiting loss trajectory. In Proceedings of the 2022 ACM SIGSAC Conference Computer and Communications Security, 2022.
Saeed Mahloujifar, Esha Ghosh, and Melissa Chase. Property inference from poisoning. 2022 IEEE Symposium on Security and Privacy (SP), pp. 1120-1137, 2022. doi: 10. 10.1109//444214 2022.9833623.
Shagufta Mehnaz, Sayanton V. Dibbo, Ehsanul Kabir, Ninghui Li, and Elisa Bertino. Are your sensitive attributes private? novel model inversion attribute inference attacks on classification models. In 3/st USENIX Security Symposium (USENIX Security 22), 2022.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning. In Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), pp. 1-15, 2018 88
E.W.T. Ngai. Yong Hu, Y.H. Wong, Yijun Chen, and Xin Sun. The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature. Decision Support Systems, 2011.
Alexandre Sablayrolles, Matthijs Douze, Yann Ollivier, Cordelia Schmid, and Hervé Jégou. White- box vs black-box: Bayes optimal strategies for membership inference. In Proceedings of the 36th International Conference on Machine Learning, 2019.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
//avvvv./.vivv///aa c abs 1409. 1556
Mary H Stanfill, Margaret Williams, Susan H Fenton, Robert A Jenders, and William R Hersh. A systematic literature review of automated clinical coding and classification systems, Journal of the American Medical Informatics Association, 1766):646-651, 2010.
Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their secrets. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022.
11

---

Published as a conference paper at ICLR 2024
Yuxin Wen, Arpit Bansal, Hamid Kazemi, Eitan Borgnia, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Canary in a coalmine: Better membership inference with ensembled adversarial queries. In The Eleventh International Conference Learning Representations, 2023.
Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Murakonda, Vincent Bindschaedler, and Reza Shokri. Enhanced membership inference attacks against machine learning models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communiccatinss Security, 2022.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In IEEE 31st Computer Security Foundations Symposium (CSF), 2018
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in Py PyTorch. arXiv preprint arXiv:2109. 12298, 2021.
A ADDITIONAL EXPERIMENTS
In this appendix we include additional results of our experiments with the Chameleon attack.
A.1 AUC AND MI ACCURACY METRICS
We report here a comparison between our attack and the existing state of the art label-only MI attacks, Gap (Yeom et al. 2018) and Decision-boundary (Choquette-Choo et al. 2021). on aggregate performance metries: the AUC (Area Under the Curve) score of the ROC (Receiver Operating Characteristic) curve and the average accuracy. Table[showe shows the values of these metrics on the three image datasets used in previous evaluations. Interestingly, Chameleon achieves superior average values in all tested scenarios.
Table Comparison of Label-only attacks on AUC and Membership Inference (MT) Accuracy metric for GTSRB, CIFAR- 10 and CIFAR-100 100 datasets. Our attack uses combination of adaptive poisoning, training shadow models (SMs) and careful selection of multiple queries (MQs) to outper- form prior attacks.
AUC
MI Accuracy
Label-Only Attack
Poison
SMs
MQs
GTSRB
CIFAR-10
CIFAR-100
GTSRB
CIFAR-10
CIFAR-100
Gup Decision- Boundary
50.6%
57.7%
73.8%
50.6%
57.7%
73.8%
0
51.5%
62.8%
84.9%
51.3%
62.4%
81.1%
Chameleon (Ours)
71.9%
76.3%
92.6%
65.2%
68.5%
85.2%
A.2 MEMBERSHIP NEIGHBORHOOD STAGE
We analyze the impact on our attack's performance by varying the size of the membership neighbor- hood set and the neighborhood threshold t. individually. Recall that the membership neighborhood for a challenge point designed compute a proxy score for our distinguishing test, which is achieved by finding candidates that satisfy Equation ( described in Section 3.2
Figure 6a we show the impact of the size of membership neighborhood on the attack success. We observe that as the size of the neighborhood increases from 16 to 64 samples, the TPR 1% FPR and AUC of our attack improves by 0.6% and 3% respectively. However, further increasing the neighborhood size beyond 64 samples does not significantly improve the TPR and AUC, as indicated by the oscillating values. Therefore, setting the membership neighborhood size to 64 samples larger should provide satisfactory level of attack performance.
Size of Membership Neighborhood.
ti) Neighborhood threshold. We now vary the neighborhood threshold tnb to observe its impact on the attack's performance. The neighborhood threshold tnb determines the selection of neighbors for the challenge point. In 66b we observe that setting t,b to 0.25 results in the highest TPR@ 1% FPR of 23.5%, while increasing tab to 1.75 decreases the TPR to 17.3%. This aligns with intuition that
12

---

Published as a conference paper at ICLR 2024
25
TPR@1
AUC
22-
21.0
PP181NFM
75.4
TPR
133
22.
23.5
10-
75.2
23
AUC
751
02
0.4
0.5
0.3
1.0
3.2
24
2.6
Number Neighbors
Neighborheod Threshold
(a) TPR and AUC by varying size of Membership Neighborhood.
(b) TPR and AUC by varying neighborhood threshold tab-
Figure 6: Ablations for Membership Neighborhood stage on C-FAR-10 dataset. We provide experiments hyperparameters used in this stage: size of the membership neighborhood, and membership threshold
systematically including samples in the membership neighborhood that are closer to the challenge point improves the effectiveness of our attack compared choosing distant samples or using arbitrary random augmentations. Additionally, we observe decrease in the AUC score when introducing distant samples in the membership neighborhood. However, this decrease is relatively more robust compared to the TPR metric.
A.3 DATA MODALITIES AND ARCHITECTURES
Table Effectiveness of Chameleon on Purchase- 100 (P-100) and CIFAR-1O (C-10) datasets over various model architectures. Our attack achieves high TPR values across various model architectures.
True Positive Rate (TPR)
Modality
Dataset
Model Type
@0.1 @PRR
IEFPP
@5%FPR
@ 10%&FP
AUC
MI Accuracy
Tahular
P-100
I-NN
8.7%
34.8%
62.2%
72.9%
92.0%
83.8%
2-NN
18.3%
45.8%
76.9%
88.8%
95.9%
89.6%
Image
C-10
Rc
8.2%
22.8%
34.7%
42.8%
76.2%
68.5%
ResNet-34
9.1%
24.6%
35.4%
43.1%
76.6%
69.2%
ResNet-50
7.4%
23.3%
33.9%
41.6%
76.3%
69.8%
VGG-16
8.6%
29.4%
49.1%
59.3%
78.6%
75.4%
We evaluate here the behavior of our attack different data modalities, model sizes and architectures.
For our experiments on tabular data, we use the Purchase-106 dataset, with a setup similar to Choquette-Choo et al. (202T): one-hidden-layer neural network, with 128 internal nodes, trained for 100 epochs. We do not use any augmentation during training, and we construct neighborhood candidates by flipping binary values according Bernoulli noise with probability 2.5%. Here, apply the same experimental settings as detailed in Section 4.1 with 500 challenges points and m 8, and a slightly lower tp 0.1. The first row of Table reports the results of the attack this modality. Interestingly, despite the limited size of the model, we observe high AUC and relatively high TPR at 1% FPR.
To observe the effect of Chameleon for different model sizes we experimented with scaling the dimension of both the feed-forward network used for Purchase-100 100 and the base ResNet model used throughout Section[4] For the feed-forward model we added single layer, which was already sufficient to achieve perfect training set accuracy, due to the limited complexity of the classification task. On CIFAR-10. we compared the results ResNet 18, 34 and 50, increasing the number of trainable parameters from 11 million to million. For the larger models we increased the training epochs from 100 to 125. Finally, we considered VGG-16 (Simonyan & Zisserman (0155) architecture pre-trained on ImageNet, which we fine-tur tuned for 70 epochs. This is a considerably larger model, with roughly 138 million parameters.
13

---

We assume the loss function L to be 0-1 loss function, so Published as a conference paper at ICLR 2024
We observe generally similar attack performance on the ResNet models, as shown in Table The largest model for both modalities, instead, showed significantly higher TPR values at low false positive rates such as 0.1% and 1%. This trend can be attributed to the tendency of larger models to memorize the training data with greater ease.
A.4 DIFFERENTIAL PRIVACY
Table 4: TPR at various FPR values for our Chamelcon attack when models are trained using DP-SGD on CIFAR-10 dataset. Differential Privacy significantly mitigates the impact our attack but also adversely impacts the model's is aceuracy.
Privacy Budget
Model Accuracy
TPR@ I I&FPR
TPR@5%IPR
TPR @ 10%EPR
AUC
MI Accuracy
20 (No DP)
84.3%
22.6%
34.8%
42.9%
76.8%
69.3%
100
57.6%
6.1%
14.8%
26.7%
61.8%
59.7%
== 50
56.8%
0.0%
11.7%
21.1%
58.9%
57.9%
32
57 2%
0.0%
9.9%
18.9%
58.8%
58.3%
16
56 56.4%
0.0%
0.0%
14.2%
55.8%
56.0%
54.8%
0.0%
12.6%
53.7%
53.7%
49.4%
0.0%
11.1%
52.4%
52.6%
We evaluate the resilience our Chameleon attack against models trained using DP-SGD (Abadi et al., 2016). We use PyTorch's differential privacy library, Opacus (Yousefpour et al. 2021). to train our models. The privacy parameters are configured with c values of [4, 16, 32, 50, 100, o0) and 5 = 10-5 alongside clipping norm of C' 5. Our training procedure aligns with that of previous works (Kurakin et al., 2022 De et al. 2022). which involve replacing Batch Normalization with Group Normalization with group size set to G 16 and the omission of data augmentations, which have been observed to reduce model utility when trained with DP. In Table 4 we observe that as e decreases, attack success also degrades showing that DP is effective at mitigating attack. However, we also observe that the accuracy of the model plummets with decrease in e. Thus, DP can be used defense strategy, but comes at high expense in model utility.
B ANALYSIS OF LABEL-ONLY MI UNDER POISONING
Let D denote the distribution from which samples 21, ..,.... Bn are sampled in iid manner. For classification based tasks, a sample is defined as 2; = (I;, where x; denotes the input vector and Yi denotes the class label. We assume binary membership inference variables m1..... m, that drawn independently with probability Pr(m; 1) X, where samples with m, = 1 are a part of the training set. We model the training algorithm random process such that the posterior distribution of the model parameters given the training data 0 21...... 2n. Mı..... My satisfies
Pr Pr(O Zn, M/. m,,) e En L(0,=.
where т and L denote the temperature parameter and the loss functions respectively. Parameter corresponds to the case of the Bayesian posterior, + 0 the case of MAP (Maximum Posteriori) inference and a small denotes the case of averaged SGD. This assumption on the posterior distribution of the model parameters have also been made in prior membership inference works such as Sablayrolles et al. 2019) and Ye et al. (022). The prior on f is assumed to be uniform.
of 21 (#1, V1), for multi-class classification task. Let C' denote the total number of classes in the classification task. We introduce poisoning by creating poisoned dataset Dp which contains l poisoned replicas of z (x1, vf), where ví * y1- Note that, all poisoned replicas have the same poisoned label vf that is distinct from the true label. The posterior Equation (2) can then be re-written as:
Without loss of generality, let analyze the
Pr(Op) F1...., Zno mi...., My: Dp) x e + Eis m, ( (((0)=)))
where term à (App = denotes the sum over all the loss terms introduced by & poisoned repli- cas. Furthermore, we gather information about other samples and their memberships in set = (z2.... ... Zn m....., m, }. that can perform concrete analysis of the loss in Equation
14

---

Published as a conference paper at ICLR 2024
Assumptions. We now explicitly list the set of assumptions that will be utilized to design and analyze our optimal attack.
The posterior distribution of the model parameters given the poisoned dataset satisfies Equation where the loss function I. is assumed to be 0-1 loss function.
The prior on the model parameters follows a uniform distribution.
The model parameter selection for classifying challenge point 21 is only dependent on 21, its membership y and the poisoned dataset D,. We make this assumption based on the findings from Tramer et al. (2022). where empirical observations revealed that multiple poisoned models aimed at inferring the membership of 21 had very similar logit (scaled confidence) scores for challenge point 21. This observation implied that poisoned model's prediction on was largely influenced only by the presence/absence of the original point 21 and the poisoned dataset Dp-
-
Poisoning impact challenge point classification. Recall that interested in analyzing how the addition & poisoned replicas influences the correct classification of the challenge point 21 label yı, considering whether 21 is member event (p(21 = y1 as selecting parameter 0p that correctly classifies 21 as y1- More formally, we can write the probability of correct classification as follows:
non-member. Towards this, define the
Theorem B.1. Given the sample 21, binary membership variable my poisoned dataset D, and the remaining training set T.
-k/7 Pr(00((1) -y1 m,, e e-mi/T k/T + 2C
()- (k+ms)/T
Proof. Let us first consider the OUT case when 21 is not a part the training set, i.c. m. = 0. Formally write the probability of selecting parameter that results in classification of sample 21 as label y1 as follows:
Pr(0)(21) = vılæ,, ma =
Based on our assumption that under the presence of poisoning, the model parameter selection for classification of 21 depends only on 21, and Dp. we can write
Pr(0,(=1) y 21, my 0,T, Dp) Pr(0,(=1)= yn 21, m. 0, Dp)
(5)
Subsequently, we can use Equation reformulate Equation 5 follows:
k/T
Pr(0,(21) v1l=1, O, e + (C 1)e-
(6)
In this equation, the numerator -//r denotes the outcome where parameter 0p is chosen resulting in classification of 2 as y1- Similarly, the terms e and (C 1)e-k/r in the denominator represent the outcomes where parameter is selected such that it classifies 2 as v and any other label except vh. respectively.
Similar to Equation we can formulate an equation for the IN case as follows:
k/r Pr(O,(z1) - 121, mi 1,) 1-1 + +e k/T + +(C - -
(k+1)/
Similar to Equation (b) the numerator e -k7 denotes the outcome that classifies sample 21 y1 The
terms e and 47+ in the denominator represent the outcomes when sample æ is classified as vf and 1/1 respectively. Term (C - 2)e-((+++1)) (k++1// denotes the sum over all outcomes where sample 21 is classified as any label except y/1 and yf- Now, by combining Equation (6) and Equation (7). we arrive
unified expression represented by Equation 4
15

---

Published as a conference paper at ICLR 2024
Optimal Attack Our goal now is to formulate optimal attack in the label-only setting that maximizes the TPR value at fixed FPR of x%, when & poisoned replicas of 2f are introduced into the training set, adhering to the list of assumptions defined earlier.
We define two events:
(2)) = y1, we say 21 "IN" the training set with probability po. else we say "OUT" with probability - Po-
If ((1)) * V1, we say is "IN" the training set with probability Pt. else we say "OUT" with probability - Pt.
can then compute the maximum TPR as follows:
Theorem B.2. Given sample 2 and training dataset that includes ls poisoned replicas of zt. The maximum TPR at 2% FPR is given as
(C-1) A/T
k/T elt (k--1)/- 1)/T (C
x' X (k-1)//T + +CC 2)--2220 -1/-1/r41 +1
X
e(k-11/77 - 2Ce
x' =(C-1)+ (0-2(000) and x' = x//00.
where probability maa
Proof.
Let x' x//100. We can start by writing the equation for FPR as:
Pr( "IN" 21, m2 0, D,T,Dp) x'
We can expand the left-hand side of the equation as follows:
Pr( "IN" 21, m2 0, D,T.D,)
Pr( "IN" (2)) = 91) Pr(00(21) m1 O,T,Dp)
+ Pr "IN" (z() + v1) x Pr(0p(2) # 1/1 Z1, my = O,T,D,)
2kt + (C (
Po c (C- P1 2h (C-
At a fixed FPR x', the above equation can then be re-written as:
po x' x ((C pI x ((C -2) e/T)
(8)
We also know that the following inequalities hold 0 Po: Pi I 1. By substituting Po as a function of p1 from Equation (8) we get:
max (0. x' x'(C 1)
0,
-C 1) 2/ -1
1) S s <PIs min d'(C 1) (1, x'eh/1
(C - 2) + c/T
(C -) (C- + ck/7
Similar to the FPR equation, we formulate the TPR as follows:
Pr( "IN" 21, mn = 1, T, D,.) = Pr( "IN" (212) = 91) Pr(O, = vıl=i, my = =1,TDD,)
Pr( "IN" 0p(2)) # vn) Pr(O) + ynl=1, m. =1,T,D,)
(k-1)// + (C-2)c-1/4
= Po x 1k-11/ + (C - 2) 1/7 +1
+ P1 (k-1)// -C- 1)e-// +1
We substitute Equation g into the above equation and get:
k/T - (k-11)/+ + - -C-2)) - 1//)) x' X c(k- 1)/T (C 2)e 1/- +1 Pr
(C- k/T
(9)
c(k-1)/+ (C 2C2 1/T +1
16

---

Published as a conference paper at ICLR 2024
The goal is maximize the above TPR equation when FPR x'. We can then write Equation 9 as a constrained optimization problem as follows:
1)
e/T e(k-1)/7 + (C - 22 -e-1/7)
max Pt
s' x e(k-1)//T + (2)--)
1/- -1
pr e(k-1)/T + (C 2)e 2)- +1
(10)
s.t. maz 0, x'(C 1'C +' +' SPi S min ( (C- ek/T
(1, x'Ek/T
(C- eh/i
In Equation 10). we observe that the first term is a constant and the coefficient of p1 is positive. Consequently, we must set P1 its minimum possible value in order maximize the TPR value.
Thus the TPR equation can be re-written as:
1C-1)+ =x'x elk 1)/T + (C 1//- DX
el - e(k-1)/T + (C e
(11)
c(1-11/T + (C - 2)e +1
where probability p = max p (o. x' ='(C-1)ta''tlzz (C-2)+et//
As previously shown in Figure (Sectionn 3.3 we plot the TPR as a function of the number of poisoned replicas for our setting using Theorem B We set the temperature parameter to a small value r = 0.5, the number of classes C 10. We fix the FPR to 5% and plot our theoretical attack. In order to validate the similarity in behavior between our theoretical model and practical scenario, we run the static version of our label-only attack where we add poisoned replicas for challenge point in CIFAR-10 dataset. We observe that the TPR improves with increase with introduction of poisoning and then decreases as the number of poisoned replicas gets higher. Note that, the assumptions made in theoretical analysis do not hold in the absence of poisoning (k- 0). Hence, we a discrepancy between the practical and theoretical attack at k = O.
C HANDLING MULTIPLE CHALLENGE POINTS
Adaptive Poisoning Strategy. We now explore how to extend Algorithm [ from Section 3.1 to
infer membership on a set of n challenge points. One straightforward extension is applying Algorithm [sepparttly to each the n challenge points, but there are several drawbacks to this approach. First, the number of shadow models grows proportionally to the number of challenge points n. rapidly increasing the cost and impracticality of our method. Second, this method would introduce poisoned replicas by analyzing each challenge point independently, overlooking the potential influence of the presence or absence of other challenge points and their associated poisoned replicas in the training data.
Consequently, we propose Algorithm 2 which operates over multiple challenge points by training a fixed number of shadow models. In Algorithm 2 (Step 3), we start with constructing 2m subsets by randomly sampling half of the original training set. These subsets provide various combinations of challenge points, depending on their presence absence in the subset, helping us tackle the second trawback. We then use these 2m subsets to iteratively construct the poisoned set Dp and train 2m shadow models per iteration (Steps 4-16). Thus, the total number of shadow models trained over the course of Algorithm is 2 2(kmax + 1) 1)m, where and kmax are hyperparameters chosen by the attacker and not dependent on the number of challenge points n. fact, the cost of Algorithm is only constant factor 2x higher than Algorithm I that was originally designed for single challenge point. Later in Appendix we analyze the impact on our attack's kss success by varying hyperparameters m and kmax-
Membership Neighborhood. Recall that, to construct the membership neighborhood for each challenge point, the attacker needed both IN and OUT shadow models specific that challenge
17

---

Published as a conference paper at ICLR 2024
Algorithm 2 Adaptive poisoning strategy set of challenge points
Input: Dataset Dadv. set of challenge points S. = ((1,,)))..... (Io, Un)) Dadv. set of poisoned points Sp {(11, vi)...... (Fng V/n) such that Vig[1.n) # yis poison threshold tp and maximum iterations Kmax. m number OUT models to train.
1: Set of poisoned replicas counters Sk {k1 O....,
Kn 0}.
Initialize a set {b) O,... bn 0} to keep track of break condition.
3: Construct subsets D;, with i E [1, 2m), by randomly sampling half of Dadv-
4: For O..... kmax do:
5: Construct poisoned dataset Dp such that [ie[ Dp contains k; replicas of (Iaa v.)).
6: Train 2m models ( 2m } on dataset D, U Die(1,2m)
For ....... n do:
8:
OUT subset of m models whose training data did not include challenge point
Query 2; on the OUT models and obtain model confidences {c ..... cm
Compute the mean 14: Σ
10:
11:
If (4, tp:
12:
Set b;- = 1.
13:
Else:
14:
h; = ki + 1. 2i=1 bi n
15:
If
break Output: Set of number of poisoned replicas Sk-
16:
point. However by design of Algorithm 2 we can now repurpose the 2m models (m IN and m OUT) trained during the adaptive poisoning stage to build the neighborhood. Consequently, there is no need to train any additional shadow models, making this stage very efficient.
D ATTACK SUCCESS AND COST ANALYSIS
We perform detailed analysis the computational cost of attack and its implications our attack's s success. Recall in Appendix C. we determined the total number of shadow models to be 2(kmax 1)m, where m and kmax denote the hyperparameters in Algorithm 2 In Table 5 vary these parameters and observe their effects on our attack's ' success and the number of shadow models trained upon the algorithm's 's completion. Each entry in Table represent tuple indicating TPR @ 1FPP and the total shadow models trained, respectively. The results are presented for kmax 4, as Algorithm terminates early (at Step 15) when kmax 2
Table 5: Evaluation of attack success and computational cost for Chameleon attack CIFAR-10 dataset by varying hyperparameters m (number of OUT models) and kmax (maximum iterations) given in Algorithm Each entry is presented tuple, indicating TPR @ 1%FPR and the total number (ResNet- 18) shadow models trained at the completion of Algorithm 2
CIFAR-10
m 1
m
YI1
m
Kmax 1
(0%, 2)
(0%, 4)
(1.1%, 8)
(1.1%, 16)
kmax 2
(13.2%, 6)
(15.6%, 12)
(20.7%, 24)
(22.4%, 48)
kmox 3
(13.2%,
(15.8%, 16)
(20.9% 32)
(22.8% 64)
Kmax 4
(13.6%, 10)
(15.9%, 20)
(21.1%, 1%, 40)
(22.9% 80)
We observe that our attack attains a notable TPR of 22.9% when setting m = and A: = 4, albeit at the expense of training 80 shadow models. However, for the attack configuration with = 2 and = 3, Chameleon still achieves a high TPR of 15.8% while requiring only 16 shadow models. This computationally constrained variant of our attack still demonstrates TPR improvement of 12.1
18

---

Published as a conference paper at ICLR 2024
over the state-of-the-an Decision Boundary attacks. Consequently, in computationally restrictive scenarios, a practical guideline would be to set m = and k =
Interestingly,
computationally expensive variant (m = 8 and k 4) still trains fewer models than the state-of-the-ar confidence-barl based attacks like Carlini et al. (2022): Tramer et al. (022)). which typically use 128 shadow models (64 IN and 64 OUT).
We also conduct a cost analysis on the more complex CIFAR-100 100 dataset, as presented in Table We observe similar TPR improvement of 13.1x over the Bouuddryy attack, while training as few as 12 shadow models. With the CIFAR-100 1-00 dataset, our algorithm terminates even earlier at kmax 2, requiring fewer models to be trained.
Table 6: Analyzing Chameleon attack success and computational cost on CIFAR-100, varying hyperparameters m and Kmax (Algorithm 2). Entries represent TPR@ @ 1FPP and the total number of (ReseNet-18) shadow models trained.
CIFAR-100
2
8
Kmax
(33.8%,
(43.7%, 4)
(50.3%, 8)
(51.1%, 16)
Kmax =
(37.2%, 6) (47.2%, 12)
(50.7%, 24)
(52.5%,
Though prior Decision-Boundary attacks (Choquette- Choo et al., 2021: Li & Zhang 202I) do not train any shadow models, they do require a large number of queries (typically 2,500+ queries) to be made to the target model per challenge point. On the contrary, after training 2(kmax + 1)m shadow models for a set n challenge points, our attack only requires atmost 64 queries per challenge point the target model. This makes attack 39x more query- than prior attack.
Query Complexity:
Running Time: We present the average rumning time for both attacks, machine with AMD Threadripper 5955WX and a single NVIDIA RTX 4090. We run the Docision-Boundar (DB) attack using the parameters provided by the code in Li & Zhang (021) on CIFAR-10 dataset. It takes 29.1 minutes to run the attack on 500 challenge points while achieving TPR of 1.1% .1% (@ 1%FPR).
For our attack, training each ResNet-18 18 shadow model requires about 80 seconds. That translates to 106.7 minutes of training time for the expensive configuration of m=8 and k=4 but achieves a substantial TPR improvement of 20.8x compared to the DB attack. Conversely, the computational restricted version of our attack (m=2 and k=3) takes only 21.3 minutes of training time while still achieving significant TPR improvement of 12. x over the DB attack.
Note that, the membership neighborhood stage in our attack requires only access to the non- non-poisoned shadow models, allowing it to run in parallel as soon as the first iteration (Step 4, Algorithm [2) of the adaptive poisoning stage concludes, incurring no additional time overhead. Our querying phase takes approximately a second to query 500 challenge points and their respective neighborhoods (each of size 64).
E PRIVACY GAME
We consider privacy game, where the attacker aims to guess if challenge point (x. ~ D is present in the challenger's training dataset D The game between the challenger and attacker A proceeds as follows:
1: C samples training data Dtr from the underlying distribution D.
2: C randomly selects b {0, 1}. If b such that (x, y) & D.r Else, samples (x, y) from D uniformly at random.
samples a point (z, 3) D uniformly random,
3: C sends the challenge point (x. v) A.
4: A constructs a poisoned dataset Dp and sends it to C.
5: C trains target model 0, on the poisoned dataset D,r U UDp
6: C gives A label-only access to the target model 0,-
7: A queries the target model 6,, guesses bit i and wins if i b.
19

---

Published as a conference paper at ICLR 2024
The challenger C samples training data Dir D from underlying data distribution D. The attacker A has the capability to inject additional poisoned data Dp into the training data Dr- The objective of the attacker is to enhance its ability infer if specific point (I, y) is present in the training data by interacting with model trained by challenger C on data Der U Dp- The attacker can only inject Dp once before the training process begins, and after training,
only interact with the final trained model to obtain predicted labels. Note that, both the challenger and the attacker have access to the underlying data distribution D, and know the challenge point (I, y) and training algorithm T, similar to prior works (Carlini et al. 2022
Tramer et al. 2022 Chen et al. 2022
Wen et al. 2023
20


Reviews:
[{"review_id": "cd9KEWGCAU", "reviewer": "Reviewer_uopX", "summary": "This paper targets at Membership Inference (MI), in which an attacker seeks to determine whether a particular data sample was included in the training dataset of a model. In contrast to the most of work in this area, the paper considers a less favorable setting: the attacker has access only to the predicted label on a queried sample, instead of the confidence level. I think this is an important problem, which should be interesting to the communities of both DP and privacy attack. To address this challenge, the paper proposes a new\nattack Chameleon that leverages adaptive data poisoning to achieve better accuracy than the previous work.", "questions": "Please refer to the section above.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a new attack Chameleon that leverages adaptive data poisoning to achieve better accuracy than the previous work.\n2. The paper observes an interesting phenomenon: for different challenge point, the sweet spot of the number of samples needed in the data poisoning is different. The paper also proposes a theory to reflect this phenomenon.\n3. Various experiments have shown the advantages of the new method.", "weaknesses": "Although the attack and the observation is interesting, I think the paper has the following weak points:\n\n1. Time complexity. Clearly from Algorithm 1, to run the adaptive poisoning, the attacker has to run the training model much more times than the baseline algorithms, making the proposed algorithm less practical. However, the paper touches little about this topic, and does not provide any comparison in the experiment section. I think this information is crucial for the readers to better understand and appreciate the proposed algorithm.\n\n2. Multiple challenge points. Usually in practice, the attacker needs to attack multiple challenge points instead of the only one. Although the paper briefly discusses this in the appendix, I think it is far from enough. Specifically, Algorithm 2 is just a simple generalization of Algorithm 1, neglecting many interesting and important problems due to more than one challenge points. For example, the problem of time complexity becomes even worse. Furthermore, due to the correlations of different challenge points, it is not clear how Algorithm 2 performs. Considering an extreme case when there are two challenge points opposing each other, it is possible after k_max iterations, the algorithm can not find meaningful k_i for both points simultaneously.\n\n3. Clarity (minor points). The paper needs to improve the clarity. For example, many definitions are used without being defined, e.g., LIRA, challenge point, in+out model. It is better to provide those definitions in the preliminary to make the paper more self-contained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5XRzf8pXrR", "reviewer": "Reviewer_ECEy", "summary": "The key contribution of this paper is to present a poisoning strategy to enhance the success of label-only membership inference attacks. The paper first shows that an existing poisoning regime negatively impacts the label-only attack's success and proposes a new way to calibrate the number of poisoning points to inject. And then, the paper proposes a way to construct shadow models and perform membership inference. In evaluation, the paper demonstrates that poisoning can increase the TPR by an order of magnitude while preserving the model's performance. The paper also analyzes the impact of attack configurations and further tests if (and also shows) DP reduces the attack success.", "questions": "My questions are in the detailed comments in the weakness section.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper presents a new poisoning attack for enhancing label-only MIs.\n2. The paper shows the poisoning can increase the attack success by 18x.\n3. The paper is well-written", "weaknesses": "1. The poisoning seems to be a straightforward adaptation of Tramer et al.\n2. The proposed label-only MI seems to be impractical.\n3. (Sec 3.3) The claim about \"theoretical\" attack is unclear.\n\n\nOverall comments:\n\nI agree it is a nice extension of existing work (Tramer et al.) to label-only settings. At the same time, this attack itself and the poisoning strategy are not surprising. So, my impression of this paper is slightly below the acceptance threshold. But if there are surprising factors that I've missed, I am not willing to fight for rejection.\n\n\n**[Straightforward Extension]**\n\nOf course, existing poisoning could not work well against an adversary who only observes hard labels. The adversary cannot \"exploit\" the impacts of poisoning until there is a change in the target's label. If too many are injected, the attacker may not know whether the target is a member. So, in the label-only settings, the key is to calibrate the number of poisoning samples. It is therefore not surprising in Section 3.2 that an \"adaptive\" poisoning strategy is needed.\n\n\n**[Practicality of This Poisoning]**\n\nHowever, I believe that choosing the right threshold $t_p$ is more challenging than shown in this paper. The paper assumes that the adversary can know the \"underlying distribution.\" \n\nBut considering that the label-only attacks are for studying the practicality in the \"true black-box\" settings (e.g., hard-labels), I wonder how well this attack can perform when there's a slight distributional difference between the training data an adversary uses and the victim's. Indirectly, the ablation study shows the proposed label-only attack is a bit sensitive to the choice of a poisoning threshold. \n\nIn practical scenarios, when a practitioner wants to check the risks of \"practical\" label-only membership leakage, the proposed attack may not be a useful one to use.\n\n\n**[Theoretical Attacks (Sec 3.3)]**\n\n(1) In most cases, the theoretical analysis means the best possible attack that an adversary can perform under a specific attack configuration. But I am not sure whether the paper presents the same.\n\n(2) I am a bit unclear on how the paper theoretically analyzes the impact of poisoning samples on the leakage. It depends on many factors, such as the training data and/or the choice of a model and a training algorithm.\n\nI think the section could a bit mislead readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xnmxL4FtBP", "reviewer": "Reviewer_JKci", "summary": "The paper proposes a data poisoning attack, called Chameleon, to enhance the privacy leakage due to label only membership inference attacks. It first shows that current attacks that aim to enhance privacy leakage via poisoning are not effective in label-only MIA threat model. The attack fails because after poisoning both the IN and OUT models misclassify the target samples. To improve the attack efficacy, Chameleon tailors the number of replicas of poisoning samples for each challenge sample. The paper shows that such poisoning significantly improves label-only MIA accuracy especially at low FPRs.", "questions": "The attack proposed is very elegant in that it is easy to implement and outperforms prior attacks. Also, the explanation of the attack is  clear and easy to understand. The paper also does a fair job in evaluating their proposed attack. Overall I think this is a good paper, but  I have the following concerns:\n\nAttack computation cost and utility:\n- Chameleon is an expensive attack given the number of models one has to train to find the right number of poisoning sample replicas. Can authors discuss the compute cost involved? I didn’t see any discussion in the main paper.\n- Given the high computation cost and the fact that modern ML model architectures are generally huge, I wonder where will this attack be useful? Which type of adversaries can afford it? It will be good to clearly discuss these aspects.\n\nSome concerns about the evaluations\n- For C100, Chameleon adds on average 0.6 replicas of poisoning samples, which means there are 40% data which need no poisoning. This means MIAs without any poisoning should work well. But this is not reflected in Table 1 results. Clarify.\n- Minor: Given that modern ML systems have generally very large and multimodal models, it might be useful to have evaluations on large and/or multimodal models.\n\nClarity of paper:\n- Figure 1 is not readable: I could not understand what it is trying to convey. Please clarify\n- Theoretical attack section currently does not clearly explain what is the attack and why this analysis is needed if the same conclusions can be drawn from empirical analysis.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Chameleon idea is elegant and easy to implement\n- Intuition and other aspects of the attack are well explained", "weaknesses": "- Chameleon is an expensive attack\n- I am not sure how will such attack be useful in practice due to the computations involved\n- Some parts of the paper need to improve presentation, e.g., theoretical attack and figure 1", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)