Title: Practical Bayes-Optimal Membership Inference Attacks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Membership Inference Attacks

Privacy

Security

Graph Neural Networks

Machine Learning
Abstract: We develop practical and theoretically grounded membership inference attacks (MIAs) against both independent and identically distributed (i.i.d.) data and graph-structured data. Building on the Bayesian decision-theoretic framework of Sabrayolles et al., we derive the Bayes-optimal membership inference rule for node-level MIAs against graph neural networks, addressing key open questions about optimal query strategies in the graph setting. We introduce BASE and G-BASE, tractable approximations of the Bayes-optimal membership inference. G-BASE achieves superior performance compared to previously proposed classifier-based node-level MIA attacks. BASE, which is also applicable to non-graph data, matches or exceeds the performance of prior state-of-the-art MIAs, such as LiRA and RMIA, at a significantly lower computational cost. Finally, we show that BASE and RMIA are equivalent under a specific hyperparameter setting, providing a principled, Bayes-optimal justification for the RMIA attack.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a theoretically grounded and practical approach to membership inference attacks (MIAs) for both i.i.d. and graph-structured data. Building on a Bayesian decision-theoretic framework, the authors derive a Bayes-optimal inference rule for node-level MIAs on graph neural networks (GNNs), addressing a previously unresolved question in privacy auditing. To render the intractable Bayes-optimal rule usable in practice, the paper introduces BASE (for i.i.d. data) and G-BASE (for graph data) as efficient approximations. These methods outperform or match existing state-of-the-art approaches like LIRA and RMIA, while requiring significantly fewer computational resources. The authors also reveal that RMIA is a special case of BASE under specific conditions, thus offering a principled interpretation of RMIA.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper derives a novel Bayes-optimal MIA for graph-structured data, filling a theoretical gap in the literature. The generalization of from i.i.d. to graph settings is significant and well-justified.\n2. The proposed approximations BASE and G-BASE are computationally efficient and outperform existing methods in many settings, making them attractive for real-world privacy auditing.\n3. The paper is well-organized with clear definitions, formulations, and diagrams. The step-by-step derivation of the Bayes-optimal rule and approximations is particularly helpful.\n\nWeaknesses:\n1. The assumption that the adversary has access to the full graph G may limit the applicability in real-world scenarios, especially for GNN attacks. This assumption should be more thoroughly discussed or relaxed.\n2. The empirical performance of G-BASE may depend on hyperparameters such as the number of shadow models and sampling strategies. More discussion on robustness to such factors would be helpful.\n3. While the work has implications for data privacy, the broader ethical and societal implications (e.g., how adversaries might exploit such tools) are not sufficiently addressed.\n\nQUESTIONS:\n1. In Section 4.1, the adversary is assumed to know the full graph G. How realistic is this in practical scenarios, especially for large or proprietary graphs? Could the method be adapted to a setting with partial or noisy knowledge of G?\n2. While the paper claims BASE and G-BASE are more efficient than RMIA/LIRA, a direct analysis of computational cost (e.g., wall-clock time or FLOPs) is missing. Including such data would strengthen the practical contribution.\n3. The performance of G-BASE under different shadow model counts or sampling methods is briefly mentioned. Could the authors provide more systematic analysis or guidance on selecting these parameters?\n4. The work focuses on node-level MIAs for GNNs. Could the Bayesian framework be extended to edge- or graph-level MIAs? If so, how might the challenges differ?\n5. Given the probabilistic nature of the Bayes-optimal rule, have the authors explored confidence calibration of their predictions? This could have practical value in privacy auditing tools.\n\nLIMITATIONS:\nPartially addressed. While the authors acknowledge the intractability of the Bayes-optimal rule and propose efficient approximations, the discussion of potential negative societal impact (e.g., how such attacks may be misused) is limited. I encourage the authors to elaborate on possible misuse scenarios and mitigation strategies, especially in light of GDPR concerns.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BASE and G-BASE, two membership inference attacks (MIAs) for iid and node-level graph data, where an adversary aims to identify whether a specific data point is included in the training set of a given trained model. These methods are developed by approximating the Bayes optimal membership inference rule. The evaluation on common graph benchmarks shows that these methods work better or on par with the statistics-based methods like LIRA or RMIA, most importantly, the authors claim that the developed methods have a lower computational cost.\n\nSTRENGTHS AND WEAKNESSES:\nDetailed comments are put under questions. Strengths and weakness are summarised below: \nStrengths\n1. Extending the Bayes optimal inference rule from [1]  to graphs is novel and important. The existing MIAs for graphs are only classifier-based methods. \n2. The evaluation is extensive on the most common graph benchmarks and GNNs.\n\n\nWeaknesses\n1. Paper clarity can be improved, specifically in the technical aspects as mentioned in the questions.\n2. The applicability of the baselines used for the analysis from the iid domain (LIRA and RIMA) to graphs is not clear.\n3. CRITICAL: There is a potential formatting issue in 228, violating double-blind policy (see formatting issue below). My current score does not account for this violation, and I will let AC/SAC/PC decide on it\n\nQUESTIONS:\nMajor concerns/questions:\n1. In the approximation of the integral (equation 8), $S_L$ function requires $\\tilde{M}$ and not $m_v$. It is unclear how $m_v$ is used here. Please clarify the substitution.\n2. In definition 2 of G-BASE attack, the dependence on index i in $\\tilde{M}$ and in $S_L$ appears to be missing. The current definition, although implemented and it seemed to work experimentally, clarification is needed on whether the sampling is correctly reflected in the current form. \n3. The posterior is approximated with its prior in line 218. How to pick an appropriate prior $p(\\phi \\mid \\mathcal{G})$? The choice is not trivial, and a discussion on it with practical justification is needed.\n4. It is not clearly explained how LIRA and RMIA (methods for iid data) are evaluated for graphs in the paper. \n5. Line 308: it is not clear why a different number of sampled graphs is chosen and not equal to the number of shadow models?\n6. Line 317: a new variable $\\alpha$ is introduced, but there exists no such variable in the G-BASE definition. What is this variable?\n7. From the results, it looks like there is no straight-forward dependence on $K$. Large $K$ is not necessarily enabling better performance. Given this observation, is there a strategy to choose $K$?\n8. The claim on the computational aspect of BASE and G-BASE is not substantiated by analysis (either theoretical or empirical). It is not immediately evident how BASE and G-BASE are computationally more efficient than other methods.\n9. On the experimental details: what proportion of nodes are used for training each benchmark? Does it have an effect on the data population pool assumption? Adding this detail (to Appendix A) would be helpful.\n\nMinor concerns (mainly to improve the paper quality):\n1. Acronyms like BASE, G-BASE, LIRA, RMIA, and GCN should be clearly defined at first use. Given that this paper introduces BASE and G-BASE, are these acronyms?\n2. Variable $\\alpha$ in line 317 is not defined.\n3. The AGGREGATE and UPDATE function in equation (1) should be further explained, especially for readers unfamiliar with GNNs. Adding references could be helpful.\n\nLIMITATIONS:\nThe theoretically derived Bayes optimal rule is intractable and can only be approximately solved. This is not a major limitation, as is common in the literature.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel and low-cost node-level membership inference attack for graph neural networks. The authors further extend their method to the i.i.d. data setting and establish a theoretical connection between their approach and existing methods. Experiments across six datasets demonstrate that the proposed method achieves comparable or superior performance to existing attacks under the same parameter configurations, while maintaining lower computational cost and higher robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n⦁\tBetter or equal performance with existing methods but with lower cost.\nWeaknesses:\n⦁\tThe core idea of this paper is Bayes-optimal framework, which has been previously explored on MIAs, limiting the novelty.\n⦁\tWriting quality needs improvement for clarity and precision.\n\nQUESTIONS:\nThis paper presents a promising and practical approach for membership inference, particularly on GNNs. The extensions to the i.i.d. setting and the theoretical justification of the proposed approximations are valuable contributions.\nHowever, I have a few concerns and suggestions for improvement:\n1.\tIn Section 6, while the experiments are comprehensive, the analysis lacks depth. There should be more deeper discussion, for example, why the proposed methods perform better on some datasets but not on others, and why your methods have better robustness against mismatched adversarial assumptions.\n2.\tIn Section 4.3, several sampling strategies are proposed to approximate the Bayes-optimal rule. It would be helpful to clarify how each strategy influences performance, and which specific strategies are used in your experiments.\n3.\tThe organization of the paper could be improved. Sections 2 to 5 could be made more concise to allow more space for in-depth experimental discussion and analysis in Section 6.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-04"}

comments: SUMMARY:
This paper introduces BASE and G-BASE, two efficient approximations of Bayes-optimal MIA attacks for standard and graph data. Their method BASE matches the baseline performance with far fewer queries and the graph version also outperforms previous baselines. The work is theoretically sound with experiments, but only on small datasets, casting a doubt on real world scalability.

STRENGTHS AND WEAKNESSES:
Strengths:
- The paper is well written with solid grounding in theory and backed by rigorous empirical methods. 
- They introduce a novel unified framework for Graph-data and other forms of data.

Weaknesses:
- Though the authors have shown excellent results, the scale of the datasets used question the real world use cases. Can Bayes-optimal MIA scale to real world models like SD-3.5 or FLUX image gen models which are trained across billions of image-text pairs ? It is hard to draw conclusions about the practicality when the empirical data is orders of magnitude smaller. 
- In the same vein, the method also requires training multiple shadow models which is also not viable as training good large models costs millions of dollars.

QUESTIONS:
1. Can this be verified on reasonably large datasets like ImageNet? Example tasks can be generation models trained on the dataset. 
2. Another prohibiting factor for real world deployment is the cost of querying these large models for obtaining log-probs. Would love to hear authors' take on that.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-04