Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The focus of this paper is on membership inference attacks. In particular, the paper aims at providing a framework that can help to access how different factors beyond information leakage from the model affect the performance of membership inference attacks, and how to design attacks that cancel out the effects off other factors. More specifically, a framework for understanding the relationship between, success of membership inference attacks and information leakage is introduced. Experiments using well-known datasets on visual tasks are provided to evaluate the performance  of the proposed attacks(in terms of AUC scores. Various frameworks are discussed in Section 3.   
main_review: ** Strengths ** 
- The investigated problem is relevant and the paper is well written;
- The notation is convient enough and all important concepts are well introduced; 
- The central idea remains interesting but it is not novel enough. 

** Weaknesses **
- The contributions are not clearly stated in the introduction section. Therefore, it appears difficult to identify which are the main contributions in this paper. According the introduction section, the main contribution is the proposed framework via the connection with binary hypothesis testing (i.e., the tradeoff between Type I and Type II errors) seem to be the main contribution.  
- The main misconception underlaying this work is the lack of novelty. The proposed framework via the connection with binary hypothesis testing is not novel and has been already proposed in a series of works (e.g., among others, see https://arxiv.org/abs/2105.03875). 
- All results in the paper rely strongly on the assumption introduced in eq. (1) which is indeed to characterize the outcome of the algorithm obtained via SGD training. However, this has not been properly justified to be considered as being strong enough to validate a possible privacy audit of ML  algorithms. The authors do not provide any valide proof or formal justification for this assumption. 
- Another central problem is the lack of numerical results and in particular, comparison with competing methods for building attacks in the literature. In addition the tradeoffs between Type I and Type II error are not always reported. On the other hand, there is very little data sets and relevant architectures to validate the results. 

summary_of_the_review: Given the above comments, I believe that the paper lacks of technical novelty, numerical results and relevant comparisons with several existent methods in the literature. To conclude, I believe that the work contains some valuable ideas which deserve to be further investigated (both theoretically and practically)  but in its current state, the contributions and the presentation of the results are too short in terms of exceptions to be accepted to be published in ICLR conference.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Membership inference attacks are attacks that infer whether a given record is in the dataset of a given machine learning model or not. By definition, (the success of) such attacks is in contrast with differential privacy.

The paper proposes multiple attacks to infer membership in the data sets of machine learning models. The envisioned goal of the paper is that these attacks can provide a more complete picture about how private a machine learning algorithm is, when compared with other membership inference attacks. Hence, the ultimate goal is to provide “reports” that go beyond “just a number” to better audit the privacy of learning algorithms.

main_review: Please list both the strengths and weaknesses of the paper. When discussing weaknesses, please provide concrete, actionable feedback on the paper.

On the positive side, the paper aims to make “auditing privacy” more modular. This goal is pursued by proposing multiple different attacks for membership inference. Then, many different factors, beyond just the success rate of the attacks are compared and reported with experiments.

On the weakness side: it is not really clear how having multiple attacks of specific form can give a full picture about whether or not an algorithm is private. There is of course the possibility that none of these attacks work on a learning algorithm and another attack works. That is why the only way to really “audit” privacy is to prove it. Alternatively, one might envision being able to prove that some class of attacks are “complete” in some sense, but I doubt that we are there yet, and certainly this paper does not provide any justification as to why their set of attacks are “complete”. In general, the paper’s plans are cryptic and each page has many ambiguous sentences with unclear goals. That is also the case for the introduction. The paper also does not justify clearly why they pick certain criteria and focus on them. For example, the bullets in page 3 are not clearly motivated/explained.

Other comments:

There are quite a few typos, plz make a pass. Example “confidentiality confidentiality” “explaining about the uncertainties” etc.

The introduction says “"Below, we first briefly review the related work in this domain, explain how different works measure privacy risk, and then…”
But then the paper moves on and explains the contributions of our work. I guess that is maybe due to the change of the paper for the submission.

The paper says “The notion of privacy underlying our framework is primarily based on differential privacy,  I would say computational DP”
Note that your definition is rather an “average case, computational, indistinguishability-based” variation of DP. That is because: you pick the data set and z at random, you work with efficient (poly time) attacks and your goal is just to distinguish 2 cases. Standard DP asks for more conditions in all three categories.

The first attack writes probabilities P(theta|D) as if we are aware of the learning algorithm. Can you please be more clear that you are basically *assuming* P(theta|D) to be what you wrote in Equation 1? In other words, you are assuming to work with a specific learner, because you sate the distribution of theta given D, no?

Page 3: “The population data used for constructing the attack algorithm, and evaluating the inference game, need to be similar, in distribution, to the training data”
I don’t get this (and many similar firm judgements). Why? I agree that natural algorithms might fall into this category, but simply stating it as a general rule needs proof (and I don’t think this is true in general).

“By violating this principle, we might overestimate the privacy loss”
I think this is wrong. The whole point of DP is that the adversary might have *arbitrary* auxiliary information. So, I see no reason why having a particular auxiliary information, and winning the security game, can be a misleading indication that the scheme was not DP; quite the contrary.

As another example of sentences that I found unclear:
“The adversary knows the underlying data distribution
What does that mean? Can I sample from it?

Maybe I am missing something here, but I don't think pi(z) should show up in Equation (4). Is not z independent of \theta and D in that case?

In your experiments of the main body: you say the data set (Purchase100) but what is the learner?

summary_of_the_review: 
I agree with the general sentiment of the paper, that having a framework for auditing privacy could be helpful. But such efforts need a much more detailed and justified approach, arguing for “complete” attacks that at least “capture known attack techniques so far”. Otherwise, just focusing on 5 attacks might be dangerous as that might suggest that schemes are private while they are not. I also think that the paper needs to be much more clear in its criteria that it proposes (in addition to the success rate of the attacks) to be part of the “report” on privacy.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a hypothesis testing framework for membership inference attacks (MIAs). The framework involves different algorithms for finding the loss threshold for MIAs, i.e. if a target data point with loss less than the threshold value then it will be inferred as a member of training data. The notation of membership privacy loss is the same as in differential privacy and the proposed algorithms improves the utility of the attacks compared to prior works on benchmark datasets.

main_review: Strengths

The hypothesis testing framework is well-motivated and the formulation is solid which also covers prior work such as shadow model MIAs. The attacks based on distillation also seem noval and not discussed before. The authors also showed that the newly proposed attacks are better than shadow model MIAs. 

Weaknesses and questions:
1. Many of the attacks could still be inefficient for auditing purposes as one might need to train 1000 reference models to perform the attacks. The numbers also are only high for models that are extremely overfitting (which is also true for previous work). How could we audit models that are well-generalized?
2. The comparison and difference between this work and prior work is vaguely discussed. No in detailed comparison with [1], which also formalized MIA with indistinguishability based framework. The hypothesis testing framework in this work is very similar to the threshold adversary in [1]. I hope authors could discuss the differences in detail. No comparison to white-box models (e.g. [2]). How would this compare to the proposed framework?
Minor: related works should be discussed in the main text instead of supplementary materials.
3. No connection made between this work and differential privacy, although the indistinguishability setup is highly related. It would be nice to see how the framework translates to privacy loss in DP.
4. Minor: column names overflowed in Table 4.

References

[1] Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, Somesh Jha

[2]  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. Milad Nasr, Reza Shokri, Amir Houmansadr

summary_of_the_review: This paper is well-motivated and the proposed framework could potentially be impactful. However, there are many empirical limitations on how to apply the attacks for auditing state of the art models. Furthermore, the evaluation and comparison of baselines methods are not thorough. I therefore recommend a weak reject.
review_title: null

Meta-review: This work provides a formal framework for discussing membership inference attacks (MIA). It then examines existing attacks and proposes some new ones. The attacks are evaluated on several datasets.  The framework mostly formalizes the types of information and error types that an attack may use and is presented as the main contribution of this work. However the presented formalizations do not appear to contribute significantly beyond the existing work on MIAs. The new attacks may be of interest and, according to the presented experiments, (mildly) improve on some of the existing MIAs. At the same time, as presented, the discussion of the the benefits of the new attacks is relatively short and reviewers did not find the results to be sufficiently convincing. Therefore I cannot recommend acceptance for this work in its current form.
Decision: Reject