## 4 Experiments
### Discussion  
In this part, we analyze the primary results of the tests and discuss the findings of the experiment. Across all the experiments, utilizing DQN models outperformed synthetic data generation methods. This superiority stems from the fact that DQN models leverage the original data to train the surrogate model, thereby imbu  
Figure 8: Performance of different methods trained on CIFAR100 dataset. The figure above compares our combination method with five different DQN extensions. The X-axis indicates training budgets from 2000 to 20,000 in 2000 intervals, and the Y-axis displays the model’s accuracy. Since a budget of 3000 was utilized to train the DQN model, the total budget is from 3000 to 23,000.  
knowledge derived from the dataset. The experimental findings indicate that employing the Prioritized Experience Replay (PER) technique leads to improved performance across all the experiments. This suggests that the PER approach, which likely involves more sophisticated decision-making or optimization processes, is more effective at utilizing the training budget to improve model performance. Experiences are ranked by PER according to their temporal difference (TD) error, or the discrepancy between the expected and actual rewards. Experiences with higher TD errors are considered more valuable for learning because they indicate situations where the model's predictions were significantly off. By focusing on these experiences, PER allows the model to learn and adapt more quickly to critical aspects of the environment. By concentrating on experiences that the model currently finds most challenging, PER can speed up the learning process, leading to faster convergence towards optimal policies. This targeted learning approach can be particularly effective in complex environments where certain states and actions are more critical to mastering the task. Focusing on high-error experiences accelerates learning, Additionally, PER includes safeguards against the model concentrating too much on a small number of experiences, thereby maintaining a balance between exploring new knowledge and exploiting known information. This balance helps in achieving not only faster but also more stable learning outcomes. DDQN comes in second because it addresses the overestimation of action values that can occur with the standard DQN. By decoupling the selection of actions from their evaluation, DDQN reduces the bias in the learning updates, leading to more stable and reliable learning outcomes. This refinement helps it perform well, though it lacks the targeted efficiency improvements of PER. ERE ranks third and enhances learning by focusing on a mix of recent experiences and the entire history, gradually focusing more on recent interactions. This approach helps in adapting to changes and optimizing the model's performance over time by balancing between exploiting learned knowledge and exploring new information. However, it may not be as targeted or efficient in prioritizing critical learning opportunities as PER. The Noisy Network ranks the lowest among the tested methods. While it introduces noise into the network parameters to encourage exploration by diversifying the policy's behavior, this can also lead to less stable training and potentially poorer performance when the task requires precise adjustments based on subtle cues from the environment, as is the case in model extraction attacks. The randomness introduced might not always align with the most informative learning paths, unlike the more structured approaches of PER, DDQN, and ERE. In summary, the key to PER's potential superiority lies in its ability to prioritize learning from the most informative experiences, thus optimizing the learning process, accelerating convergence, and eventually resulting in enhanced DQN model performance in model extraction tasks. One critical aspect of dealing with CIFAR-100 is the need for a model with greater capacity.