## 5 Conclusions  
In this research, we delved into model extraction attacks, focusing on the efficiency of various DQN extensions for sample selection and synthetic data generation techniques like Jacobian-based methods, FGSM, LinfPGD, and data-free attacks for enhancing surrogate model training. Our contributions include evaluating the effectiveness of DQN extensions (DDQN, Dueling DQN, Noisy Network, Soft update, PER, ERE) for selective sample extraction, investigating both gradient-based and data-free synthetic data generation techniques for model extraction, and conducting comprehensive experiments on MNIST datasets. Our study contrasted these approaches across different query budgets to identify the most effective strategies for executing black-box attacks under limited query scenarios.  
Our findings reveal that the Prioritized Experience Replay (PER) technique consistently outperforms other methods in achieving higher accuracy during the model extraction process. Specifically, PER's sophisticated decision-making optimizes the use of training budgets more effectively. In contrast, data-free methods demonstrated significantly lower performance, likely due to an insufficient query budget for generating useful training images. Among synthetic data generation techniques, LinfPGD emerged as the most effective, systematically identifying adversarial examples that improve surrogate model performance. This highlights the importance of quality training data in model extraction attacks, where LinfPGD's methodical approach to generating adversarial examples proves crucial.  
Through comprehensive testing and analysis, our study confirms that utilizing DQN models, particularly those employing the PER approach, significantly outperforms synthetic data generation methods. This is attributed to DQN models' ability to leverage original data effectively, thereby incorporating the most informative insights into the surrogate model. The research underscores the pivotal role of synthetic data generation in scenarios where access to the original dataset is constrained, with LinfPGD emerging as the most effective method in such contexts. In conclusion, this paper sheds light on the multi-faceted strategies of model extraction attacks, demonstrating the paramount importance of sophisticated sample selection and synthetic data generation techniques in enhancing model performance. The elucidation of PER's and LinfPGD's effectiveness provides valuable insights for advancing model extraction methodologies, setting the stage for future explorations in optimizing attack strategies within constrained environments.  
Conceptualization, D.H. and S.C.; methodology, D.H. and S.C.; investigation, D.H.; validation, D.H. and R.B.; data curation, D.H. and R.B.; writing, D.H. and R.B.; review and editing, S.Z. and S.C.; supervision, S.C. All authors have read and agreed to the published version of the manuscript.  
This research was supported by the NSF through SaTC 231271 and the Vice President for Research and Partnerships and the Data Institute for Societ Challenges at the University of Oklahoma through a DISC Seed Grant Award.  
Not applicable.  
Data Availability StatementData are contained within the article.  
The authors declare no conflicts of interest.