Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on the analysis of membership inference attacks (MIAs), in  particular addressing the problem of high false positive rate (FPR) that still affects the  state-of-the-art solutions. The authors suggest an approach to MIA that is based on difficulty calibration, i.e., a method where the predicted membership score is adjusted  to the difficulty of obtaining the correct classification for the target sample, in order to reduce the FPR. As illustrated in Figure 1, previous work on the score-based membership inference attack is highly unreliable for separating easy-to-predict non-members from hard-to-predict members, as both can achieve high membership scores. This technique appears to be particularly beneficial in helping to distinguish between  member and non-member samples that are overrepresented in the data distribution. The authors support the proposed method with a large body of mostly favorable experimental results.
main_review: ** Strength ** 
- The paper is overall well written and easy to understand. The contributions are clearly stated and explained. 
- The main contribution is the application of difficulty calibration (inspired by Long et al., 2018) to MIA. Wide body of experimental  results using different datasets (only images though) which mostly support the proposed approach by showing favorable results. 
- The analysis with different member-non member ratios is definitely interesting and well done.

** Weaknesses** 
* It would be highly beneficial to better and more formally support the intuition about easy-to-predict  non-member samples page 3 section 3. Indeed, the intuition lacks a deep analysis and is only supported by most of the experimental (empirical) results. Fors instance, several fundamental questions remain unsolved such us:
- Are there further connection to the generalization capabilities of the network ? Several results have been mentioned in the literature but this aspect has not been discussed in this work. 
- It is not clear of the condition to ensure the validity of expression (6) are satisfied. This assumption should be justified and further investigated. 
- What about possible context with the accuracy of the network? Calibration and accuracy are two different concepts but they may also be related to the potential weakness to MIA. 

summary_of_the_review: In order to better understand the improvement coming with the proposed method, it would be useful to have a deeper understanding of the reason for which it is indeed beneficial. More details on the differences between the models f and g are to be provided: different random
initializations are probably involved as well as different batch shuffling. I guess that mainly the g models are going to converge faster on the training data, presenting more variance in the performance for non member data, which in turn makes it easier to find an effective threshold for the MIA problem. However, this does not seem effective on some datasets. Could the authors investigate those issues and shed more light on the relation between the type of data and the success/insuccess of their method?
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper aims to solve the issue of high false positive rates in modern membership inference attack methods by computing the difference in model prediction metrics (e.g., loss, confidence) between the target model and a reference model. The intuition is that “easy-to-predict non-members” will have a small difference between the target and the reference model, while “hard-to-predict members” will have a large difference. Experimental results on multiple datasets also validate the effectiveness of proposed method.
main_review: [strengths]
1. The idea of avoiding high false positive rates is indeed important for membership inference research.
2. Experimental results on multiple datasets indicate significant improvement than prior approaches.

[weaknesses]
1. The idea of difficulty calibration is not new. As the authors already pointed out, Long et al. (2018) and Carlini et al. (2020) already used similar technique to compare the difference (or ratio) between target models and reference models. What are new things in this paper?
2. The paper fails to compare with existing attacks which are also designed to decrease false positive rates, specifically, the following paper. --- Jayarama et al., “Revisiting Membership Inference under Realistic Assumptions”, PETS 2021.
3. Besides evaluated metrics (loss, gradient norm, confidence), recent works also leverage entropy for membership inference. It would be great if the authors can also include entropy metric as well.  --- Song & Mittal, “Systematic Evaluation of Privacy Risks of Machine Learning Models”, USENIX 2021.

summary_of_the_review: I like this paper’s focus on reducing high false positive rates. However, unless the authors clearly explain the difference with prior works (Long et al. (2018), Carlini et al. (2020)) and show the advantage over other attacks (Jayarama et al. (2021)), I suggest not accepting this paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes to improve membership inference attacks by calibrating the attack threshold on each example's hardness.
main_review: This paper proposes a natural and simple idea to improve membership inference attacks: train a few reference models on the same distribution, and then use these as a "prior" for the distribution of losses of non-members.
This idea is not entirely new: it was proposed and analyzed both in Sablayrolles et al. and Long et al. which the paper references. It would be helpful to clarify this paper's contribution with respect to these two papers. In particular, it seems to me that the attack proposed in this paper is strictly *weaker* than the one of Sablayrolles et al.
In "Connection to posterior inference", this paper claims that the proposed approach is equivalent to that of Sablayrolles et al. But this isn't quite true: Sablayrolles et al. train reference models on subsamples of the data so that they can approximate both the average loss of an example when it is a non-member *and when it is a member*. Their attack then essentially determines whether an example's observed loss is closer to the average non-member loss or the average member loss of that sample.
The approach in this paper is similar, but only considers reference models that do not contain a victim point. So this attack is necessarily weaker.

While the attack idea thus doesn't seem particularly novel, the analysis and experiments are however very thorough and nicely complement the more theoretical paper of Sablayrolles et al. The ablation studies are overall very solid and give a good idea of what factors influence the attack's strength. A few minor comments:
- Some of the model accuracies in Table 1 are fairly low (e.g., CIFAR-10/100). Do you get similar results if you use a better model or training setup?
- The finding that data augmentation leads to less privacy leakage is contrary to what was found by Choquette-Choo et al. Did you also use augmentations for the attack as in that paper?
- Figure 4 (right) and Figure 6 are hard to read as they display a 3-dimensional relationship. Instead of a full PR curve, it might be better to extract a single scalar for each experiment (e.g., the AUC, or the precision at a recall of 50% or something like that) and then plot that scalar for varying ratios in Figure 4, and for varying epsilon in Figure 6.
- The AUC curves in Figure 5 do not seem to reach a plateau. Can you further improve it by training more reference models? 
summary_of_the_review: A nice paper with an idea that is not entirely novel but with a rigorous empirical analysis.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work proposes a simple output calibration method to suppress false positive errors of membership attacks. The core idea is to define a calibrated decision score as the difference between the original membership score and the expected membership score trained by a randomized training algorithm using a shadow dataset (a dataset following the same distribution as the true training data set). The calibrated membership decision score can provide a lower false positive rate than the original membership score. 

 
main_review: Strength: Controlling false positives of membership attacks is important for practical membership inference scenarios. Research efforts along  this direction should be encouraged. 

Weakness: 
The theoretical rationality of the proposed calibration method is not provided in the study. Though it is mentioned that the proposed method is associated with posterior inference, more formal study establishing the link is still needed. The discussion at the end of Section.3 is not convincing. 

summary_of_the_review: The theoretical reasoning the performance of the proposed calibration method is not provided yet necessary.
review_title: null

Meta-review: This paper proposes a technique to improve membership inference attacks by
carefully applying "difficulty calibration" to improve the attack success
rate. The reviewers are split on this paper. They all generally agree on
the facts: the paper introduces a (somewhat) new technique and performs a
solid evaluation, but the novelty on top of prior work isn't all that high.

On the whole I believe this paper should be accepted. This paper has identified
a very clear problem with existing attacks (poor performance at low false
positive rate) and has carefully developed a way to improve on this metric. A
thorough evaluation has convinced the reviewers that this paper does what it
set out to do.

It comes down to a question of novelty then. And here the question is this:
does someone who reads this paper learn something new that wasn't obvious
before? Part of this can be novelty in the method---and I agree with the
reviewers that this paper lacks novely in the method. However the paper does
not lack novelty in the ideas on the whole. While Long et al., Sablayrolles et
al., and Carlini et al. do all use some kind of low false positive rate
evaluation and calibrate for low loss, none of these papers actually go out
of their way to evaluate this fact explicitly. And so even if this paper
had no technical contribution at all, the simple measurement study in and
of itself would be a useful insight.

Machine learning research at present focuses fairly heavily on novelty
of the techniques. While this is good, it's also important to go back and
actually evaluate what we have. That's what this paper does, and it does
it well enough to be worth accepting.

The paper would definitely be improved by following some of the advice of
the reviewers and including comparisons to prior work (e.g., especially
clarifying the relationship to Sablayrolles et al. and if it is true that
this attack is a simplification of this prior one and is thus less effective)
and I hope the authors will take the opportunity to do this.
Decision: Accept (Poster)