Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The submission titled "Effective passive membership inference attacks in federated learning against overparameterized models" describes a well-motivated black-box membership inference attack that can be instantiated in a federated learning setting, i.e. this is an inference from user update/gradient to dataset membership likelihood. The submission motivates the effectiveness of gradient alignment as a metric of membership inference success for large, overparametrized models and shows empirical success in a variety of simulated FL scenarios.
strength_and_weaknesses: I'm a fan of this work. The proposed approach is straight-forward and well-motivated and after reading this submission I am convinced that this is a strong baseline membership inference attack for federated learning. I have a few minor questions concerning ablations, notes about related defenses and minor remarks which I will list below, but overall my impression is positive.

* One defense against this attack thatis not well evaluated in the main body is aggregation. I think it would be great to consider aggregation (i.e. the server update is aggregated over more data, for example from multiple users through secure aggregation) as a central defense in Section 4, as this would be to the most likely encountered defense mechanism in practice. I see some discussion in Table 8 which I think is related, but it would be great to include a plot of the effectiveness of the attack in relation to the number of data points used to compute the attacked update.

* Concerning the validation of overparametrization: While the rank experiment in Fig. 3 is interesting, and certainly a valid case of overparametrization, to me, other experiments would be closer to the convential understanding of overparametrization: One example could be to use a ResNet model on CIFAR-100, but to scale the width of all layers equally.

* The one thing I don't understand about Section 2.3 is the summation over $a \in \mathbb{Y}$. For a conventional membership inference attack I would assume the data point to be tested to be a pair $(x,a)$ containg both image and label. Here, the label seems to be an additional unknown to the attacker? It would be great to briefly formalize the threat model some where in Section 2 and clarify this.

* Concerning ablations, the attack is introduced targeting only one layer in the model. This is ablated in Fig.2 (for what seems to be AlexNet), but I couldn't find information which layer is targeted in other settings. I see the automated choice based on non-member information, I am basically just interested in the outcome. Also, a reasonable extension of Fig.2b) for me would include the attack, but using the entire gradient, just as baseline. 
clarity,_quality,_novelty_and_reproducibility: I found the writing in this work to be clear. While the related work section in the main body is short, the authors do supplement it with a larger overview in the appendix for interested readers.

Concerning additional related work, the authors might find Haim et al, "Reconstructing Training Data from Trained Neural Networks" interesting, which describes a series of data reconstruction attacks based on gradient decompositions of trained models. Overall, for me, the connection of this attack to data reconstruction attacks is particularly interesting, for example, the proposed attack can be seen as a zero-order evaluation of the gradient inversion objectives based on cosine similarity, evaluating on the data point to be tested.
summary_of_the_review: In summary, I believe this to be a significant contribution and recommend acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes membership inference attacks in federated learning for overparameterized models. The main observation is that in later stages of training an overparameterized model, the gradients of different examples will become orthogonal and thus from the dot product between gradients, the attacker can infer the membership. The attacks were evaluated on multiple datasets and different mitigation strategies were discussed and implemented.

strength_and_weaknesses: Strengths

1. The main observation about gradient orthogonality and the use of it in membership inference is novel. 
2. Experiments covered a different range of datasets and defense methods.

Weaknesses

1. This paper is not very organized and some parts are confusing to read. For example, equation (2) is hard to parse due to the superscript, and half-step update is not clearly defined.
2. The threat model is particularly not clear. What is the capability of the attacker? What exactly does the attacker observe, the individual gradients for each client or the summed gradients overall clients? How do you motivate such a threat model? 
3. The implementation details of differential privacy were missing. It’s very surprising that Table 5 showed the ineffectiveness of DP as it is contradicting the DP theoretical guarantee. What are the privacy budgets? Did you perform the attack on the individual gradients with local DP or did you perform the attack on the aggregated gradients with central DP? Also, training with DP on datasets like CIFAR100 with less than 1% drop in utility is very hard, and the authors should provide more details to support their claims that such a defense strategy is meaningful. 

Minor: Section 2.1, federated averaging after equation 1 should be an average but currently it is a sum.

clarity,_quality,_novelty_and_reproducibility: The clarity and quality of writing needs to be improved. The use of gradient orthogonality for membership inference is novel. The detailed hyperparameters were described in the appendix but no code or pseudocode of the attacks were provided.

summary_of_the_review: I don’t think the paper is ready for publication given the weaknesses discussed above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new passive membership inference attack in federated learning against overparameterized models. The attacker has no access to the private training data information and any other data instances (e.g., shadow dataset) sampled from the same data distribution. The proposed attack method is based on the similarity between client updates and gradients of instances. The evaluations demonstrate the effectiveness of this method.

strength_and_weaknesses: Strengths:

- Trendy topic
- New perspective on membership inference
- Easy to follow

Weaknesses:

- Insufficient explanations for the intuition
- Unbalanced validation dataset for the MIA

Comments for the authors:

This paper proposes a new passive membership inference attack in federated learning against overparameterized models. The intuition for the proposed attack method is novel and it is easy to implement such attacks. The experimental results also show the effectiveness of this method.

However, I have the following concerns:

- The intuition needs more explanations. Figure 1 shows the distributions of pair-wise cosine similarity of gradients from different instances. However, it seems that there is no big difference between the distributions of members and non-members. So why the cosine similarity can serve as a feature to discriminate members from non-members? I would suggest the authors add more explanations here.

- The validation dataset is biased. The validation dataset used by the attack model only contains non-member instances. Is there any method to prevent the attack model from giving biased predictions? For example, if the attack model always predicts the data samples as non-members, this attack model will perform perfectly on the validation dataset, which does not mean that this attack model can give correct predictions to members as well. I suggest the authors give some explanations here and improve the original method if possible.

- Will the data augmentation strategies (e.g., RandomCrop) in the clients affect the attack effectiveness? I am curious about whether some data augmentation strategies will affect the similarity between the client updates and the gradients of data samples. I would suggest the authors also complement the experiments for this ablation study.

- This paper claims that the proposed method will not work well in small neural network models in Section 1. However, in Section 4, the authors also evaluate the relationship between the model overparameterization level and attack effectiveness and it seems that the attack is still effective (though with lower performance) even when the model has relatively low model overparameterization levels. Then why this method does not work well only for small models? I would suggest the authors give more analysis for it and complement the corresponding ablation study if necessary.

Minor:

- On Page 2, "iff $u \in \mathcal{O}^\prime$" -> "if $u \in \mathcal{O}^\prime$"

clarity,_quality,_novelty_and_reproducibility: good
summary_of_the_review: see strength and weakness
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: (a) The paper addresses the problem of membership inference attacks: namely, understanding whether a given data point is in the data set. Its key observation is that in over-parameterized models that are well trained, at some points gradient will become roughly orthogonal. And as a result, it will be possible to perform an attack by cosine similarity with the gradient of the queried 
member. 
(b) The interesting aspect of this paper is the link between over-parameterization and membership attacks, due to a certain orthogonal structure in the model. This seems new, and would be of interest to the federated learning and secure-learning audience. Experiments also support the results.
(c) The attack model could be made more realistic, although the reviewers thought it was acceptable for demonstrating the observations in the paper. 
justification_for_why_not_higher_score: Taking into account the strengths and weaknesses above and the reviewer comments, a poster decision seems adequate. 
justification_for_why_not_lower_score: Sufficient novelty to accept.
Decision: Accept: poster