Kati8/ICLR_2023_2025 huggingface dataset:
authors: Khanh Nguyen
authors: Raouf Kerkouche
authors: Mario Fritz
authors: Dimosthenis Karatzas
reviews: 
Comments: This paper introduce the first document-level membership inference attacks Document-Level Membership Inference Attacks (DocMIA) for Document Visual Question Answering (DocVQA) models, addressing privacy risks in multimodel contexts. Through extensive experiments across multiple datasets and models, significantly outperform existing membership inference baseline.
Strengths: 1、This paper proposes Document-Level Membership Inference Attacks (DocMIA), which deal with the multiple appearances of the same document in the training set.

2、DocMIA consistently achieves high performance across all target models, the strong performance of  DocMIA highlights the privacy risks posed by optimization-based features of membership.
Weaknesses: 1、The writing in this paper is clear and accessible, providing a well-structured and understandable presentation of the authors' ideas and methods. However, the text is overly verbose, with an excess of written content and a lack of visual aids. The absence of a diagram illustrating the attack process detracts from an intuitive understanding of the attack methodology. Adding visual elements would enhance the reader's comprehension and provide a more direct insight into the core processes discussed.

2、There are some typographical problems with the manuscript. The position of Figure 1 is not ideal and should be adjusted to be centered side-by-side, similar to Figure 2.
Recommendation: 6
Reviewer confidence: 2
Soundness/correctness: 3
Clarity: 2
reviews: 
Comments: The paper aims to make a fine-grained membership inference attach, which can determine whether a sensitive document is included in the training dataset. For the white box and black box, the authors have developed different MIA approaches according to their characteristics. Experimental results confirm that their methods outperform baselines.
Strengths: 1. This paper is the first attempt to target document-level membership inference attacks.
2. This paper designs attack modes for white box and black box respectively.
Weaknesses: 1. The challenges of the work are underdescribed by the authors, and it is difficult for the reviewer to understand the technical challenges between fine-grained inference attacks and coarse-grained attacks, proposed by Tito et al.. The authors express it more as if they were writing an experimental report.
2. "shadow training of proxy models becomes infeasible.", the reviewer suggests the authors describe in words how to solve this problem and highlight it.
3. While the paper introduces methods to adapt the attack to black-box models through knowledge transfer, this approach may not be as effective as direct white-box attacks due to the inherent limitations in approximating the behavior of complex models.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 2