Title: Data-Free Model Extraction for Black-box Recommender Systems via Graph Convolutions
Venue: NeurIPS 2025
Accepted: yes
Keywords: Recommender System

Model Security

Model Extraction Attack
Abstract: Privacy and security concerns are becoming increasingly critical for recommender systems, as model extraction attack provides an effective way to probe system robustness by replicating the model’s recommendation logic — potentially exposing sensitive user preferences and proprietary algorithmic knowledge. Despite the promising performance of existing model extraction methods, they still face two key challenges: unrealistic assumptions on the requirement of accessible member or surrogate data and generalization problem where surrogate model architecture constraints lead to overfitting on generated data. To tackle these challenges, in this paper, we first thoroughly analyze how the architecture of surrogate models influences extraction attack performance, highlighting the superior effectiveness of the graph convolution architecture. Based on this, we propose a novel Data-free Black-box Graph convolution-based Recommender Model Extraction method, dubbed DBGRME. Specifically, DBGRME contains: (1) an interaction generator to alleviate the need for member data requirements in a data-free scenario; and (2) a generalization-aware graph convolution-based surrogate model to capture diverse and complex recommender interaction patterns for mitigating the overfitting issue. Experimental results on various datasets and victim models demonstrate the superiority of our attack in data-free scenarios (e.g., surpassing PTQ data-require methods with 17.4% improvement on LightGCN). Code is available: \url{https://github.com/Vencent-Won/DBGRME.git}.


Peer reviews:
comments: SUMMARY:
This paper proposes a new method for data-free model extraction via graph convolutions. This is an adversarial attack setting in which an adversary wants to extract the model or parameters of a recommender system via system queries.

STRENGTHS AND WEAKNESSES:
-- New approach to the problem and results that seem positive.

-- This topic, while relevant enough to the venue, is fairly niche. There have only been a few papers on the problem across several years. I'm not totally convinced as to how realistic of an attack setting this is, which may be related to the obscurity of the topic.

-- Relatively small datasets, not certain about the scalability of the approach.

-- The member data rates and query rates seem to be unrealistic in an attack setting.

QUESTIONS:
-- Naive question, but what is meant by "member data rate" in what's a "data free" setting? I would have thought only queries were impossible and that data is not available, though I assume this is a misunderstanding on my part.

-- The member data and query rates seem not realistic from an attack perspective. Can you describe a realistic attack setting in which the rates presented here are reasonable? How does the method work for very small rates?

-- The datasets look fairly small. What is the scalability profile of the method on larger datasets?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on model extraction attacks in recommender systems. Considering that existing methods often rely on unrealistic assumptions about data accessibility, a Data-free Black-box Graph convolution-based Recommender Model Extraction (DBGRME) is proposed. The core of DBGRME is to make up for the need for member data through an interaction generator, and propose a Generalization-Aware Graph Convolution-based Surrogate Model to capture complex recommendation interaction patterns. Finally, the effectiveness of the proposed strategy is demonstrated on multiple datasets and recommendation models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The data accessibility and generalization that this work focuses on are two core issues of model extraction.\n2. The proposed interaction generator avoids dependence on sensitive member data.\n3. The experimental results show that the proposed method seems to be effective.\n\nWeaknesses:\n1. The superiority of graph convolution-based recommender models in the empirical experiments on three recommendation models is unconvincing. Are there similar findings on more models? Or analyze its advantages theoretically?\n2. Although the paper claims to attack under the \"black box\" setting and limited queries, it is usually difficult to obtain the top-k recommendation list in the strict real world, which limits the applicability of the proposed attack. The paper should discuss the performance of the attack under more stringent query responses.\n3. How is the query budget selected by default? From Table 6, this choice is strange.\n4. Minor: Yelp in Line 321 should be ML-1M?\n5. Are the conclusions of the ablation experiments and sensitivity analysis on the Yelp and Gowalla datasets similar? I understand the space limit, but it is recommended to add it in the appendix.\n6. Although the baseline methods in the experiments are related, it would be more convincing if they could be compared with the recent model stealing method [1].\n\n[1] Wang, Yihao, et al. \"Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 12. 2025.\n\nQUESTIONS:\nPlease refer to \"Strengths And Weaknesses\".\n\nLIMITATIONS:\nMore discussion on limitation is needed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles model-extraction attacks on recommender systems under the strictest data-free black-box setting: the adversary sees only top-k outputs and must generate all queries. The authors first run a careful diagnostic and observe that graph-convolution surrogates generalise better than architecture-matched ones. Guided by that insight they build DBGRME, which couples : 1) an interaction generator that synthesises user–item pairs and, GAN-style, maximises disagreement with the victim and 2) a generalisation-aware LightGCN-like surrogate with frozen user embeddings and an ℓ2 constraint on item updates to curb over-fitting\n\nOn four public datasets, DBGRME outperforms prior data-free baselines and, on average, even beats partial-data+query (PTQ) by 17 % Agr@50 on LightGCN. An ablation verifies that both the GA module and the constraint are useful. Overall, the attack pipeline is novel, and empirical gains are convincing.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths \n- Data-free is the worst case for defenders and mirrors scenarios where member data are protected by privacy law.\n- The initial cross-architecture study motivates the choice of graph surrogates instead of blindly copying the victim architecture, an idea that may transfer to other domains.\n- Provides an end-to-end view; gradients through discrete sampling are handled via re-parameterisation  \n- Comprehensive ablations and evaluation : Five victims spanning MF, deep MLP and GCN families, four datasets of varying sparsity, plus budget sweeps give a robust empirical picture.  \n- Ablations and hyper-params show thoughtful tuning. Code is to be released, aiding reproducibility .\n\n### Weaknesses\n\n- Query efficiency Budgets of 4-9 M queries (Gowalla/Yelp) are two orders of magnitude above typical commercial rate limits. Unsure how significant these dataset results are applicable\n- The interaction generator requires extensive training (100 iterations with 1,000 synthetic users per loop), implying over 10^8 forward passes. Is that quite expensive? \n- The paper lacks discussion of attack detectability and potential countermeasures.\n\nQUESTIONS:\n- Would be good to provide some wall clock time or compute utilisation for generator cost\n- The primary metric (Agr@50) treats all items equally, whereas practical scenarios prioritize high-rank agreement. Additional metrics such as\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper focuses on the issue of data-free, black-box model extraction in recommender systems. It presents a novel approach, DBGRME (Data-free Black-box Graph Convolution-based Recommender Model Extraction), to overcome limitations in previous methods. DBGRME utilizes two key components: an interaction generator that alleviates the need for member data and a graph convolution-based surrogate model that improves generalization and mitigates overfitting. The method is validated through experiments on multiple datasets and various victim models.

STRENGTHS AND WEAKNESSES:
Strength:

1. The introduction of the surrogate model and the interaction generator addresses the key challenge of model extraction in data-free black-box settings.
2. The paper includes experiments demonstrating DBGRME’s superior performance.

Weakness:
1. My concern is the lack of comparison with some SOTA works [1]][2][3].  
2. Is the query budget set too large compared to the existing works? 
3. Some common dataset settings (Steam and Beauty) are missing.
4. I suggest the authors add some discussions about the possible defense methods.

[1] Wang Y, Su J, Chen C, et al. Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12766-12774.

[2] Wang C, Sun J, Dong Z, et al. Data-free knowledge distillation for reusing recommendation models[C]//Proceedings of the 17th ACM Conference on Recommender Systems. 2023: 386-395.

[3] Lin Z, Xu K, Fang C, et al. Quda: Query-limited data-free model extraction[C]//Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security. 2023: 913-924.

QUESTIONS:
Please see the weaknesses.

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03