###### Abstract  
In the rapidly evolving landscape of cybersecurity, model extraction attacks pose a significant challenge, undermining the integrity of machine learning models by enabling adversaries to replicate proprietary algorithms without direct access. This paper presents a comprehensive study on model extraction attacks towards image classification models, focusing on the efficacy of various Deep Q-network (DQN) extensions for enhancing the performance of surrogate models. The goal is to identify the most efficient approaches for choosing images that optimize adversarial benefits. Additionally, we explore synthetic data generation techniques, including the Jacobian-based method, Linf-projected Gradient Descent (LinfPGD), and Fast Gradient Sign Method (FGSM) aiming to facilitate the training of adversary models with enhanced performance. Our investigation also extends to the realm of data-free model extraction attacks, examining their feasibility and performance under constrained query budgets. Our investigation extends to the comparison of these methods under constrained query budgets, where the Prioritized Experience Replay (PER) technique emerges as the most effective, outperforming other DQN extensions and synthetic data generation methods. Through rigorous experimentation, including multiple trials to ensure statistical significance, this work provides valuable insights into optimizing model extraction attacks. s +
Footnote †: journal: Medical  
A standard example of an adversarial model extraction attack appears in Figure 1. By systematically querying the model and analyzing its responses, attackers can infer valuable information, allowing them to construct a surrogate model that tightly resembles the original [10; 11]. The implications of successful black-box adversarial model extraction attacks are multifaceted, ranging from intellectual property theft and undermining the competitive advantage to enabling more sophisticated attacks, such as crafting adversarial examples or identifying vulnerabilities within the model [12]. In this research, we investigate two key aspects of model extraction attacks: sample selection and data-free strategies. Sample selection involves determining which input samples are most advantageous for querying the target model, potentially leading to more efficient and effective extraction. Conversely, data-free attacks seek to replicate a model without requiring accessibility to the training data's initial source, presenting a significant threat to privacy. For sample selection, we explore the efficacy of various extensions to the Deep Q Network (DQN) [13; 14], leveraging the combined strengths of active learning and reinforcement learning techniques. Active learning, in our approach, serves to judiciously select queries that are most likely to yield informative feedback from the victim model, thereby enhancing the training of a surrogate model, which simulates the target model's decision boundaries. At the same time, reinforcement learning refines the process of selecting queries, establishing an adaptive approach that reduces the number of queries needed while enhancing the strength of the adversarial examples created [15]. By leveraging the power of DQN and its variants, such as Double DQN (DDQN) [16], Dueling DQN [17], Noisy network [18], Soft update [19], Prioritized Experience Replay (PER) [20], and Emphasizing Recent Experience (ERE) [21], we aim to identify the most informative samples for querying the target model. Our approach involves training DQN agents to learn sample selection strategies that maximize the information gained from each query, potentially leading to more accurate model extraction with fewer queries. In addition to sample selection, we investigate techniques for generating synthetic samples to aid in model extraction. Specifically, we evaluate the performance of gradient-based approaches, including Jacobian-based methods [22], Linf-projected Gradient Descent (LinfPGD) [23], and Fast Gradient Sign Method (FGSM) [24].