# Cloud-Edge Collaborative Model Adaptation Based on Deep Q-Network and Transfer Feature Extraction
## 4 Experiment
### Consistency Verification of Cloud-Edge Collaboration Implemented Through Reinforcement Learning  
For the detection results of different scenes, the consistency of the side detection results of a certain category of aircraft and the cloud-side fusion results are updated iteratively during the training process of the DQN network. The following figure is a randomly selected picture with an angle of 0 in six scenes, and the number of aircraft is 6, 8, 12, 14, 22, and 33 respectively. Scenes 1, 2, and 3 correspond to the images shown in Figure 4, while Scenes 4, 5, and 6 correspond to the images shown in Figure 5.  
As the number of DQN iterations increases for each type of scenario, the reward function changes, as shown in Figure 6.  
The trends of the curves indicate that the DQN agent is able to quickly learn effective strategies, leading to the optimization of model performance. A detailed analysis is provided below.  
(1) Overall Characteristics of the Convergence Trend  
The reward curves in all six scenarios show a declining trend followed by stabilization, indicating that the DQN is able to converge effectively in these environments. In the initial stage, the reward values are relatively high, reflecting significant discrepancies between the edge model and the cloud fusion results before optimization. After three to five training iterations, the rewards in most scenarios approach zero, indicating that the edge model, under the guidance of the DQN, has closely approached the detection performance of the cloud model.  
(2) Observations of Inter-Scenario Variability  
In Scenarios 1 and 2 (with fewer aircraft), the reward function drops rapidly with slight fluctuations, and policy learning is nearly complete by the second iteration. This indicates that in simpler scenarios with less variability, the DQN policy converges faster, and the transfer strategy is easier to determine. In Scenarios 3 and 4 (with medium target density), the reward curves exhibit larger fluctuations but still converge after the fifth  
Figure 6: The evolution of the reward function over episodes across the six experimental scenarios.  
iteration. This suggests that in scenarios with moderate target density, the stability of policy learning remains satisfactory. In Scenarios 5 and 6 (high-density, crowded airports), the initial reward values are the highest, and the convergence process is slightly slower. This indicates that in complex scenarios, the discrepancy between the edge model and the cloud model is more pronounced, requiring the DQN to perform more interactions to learn an appropriate transfer strategy. However, convergence is eventually achieved, demonstrating the method's adaptability in complex environments.  
(3) Final Converged Value  
According to the figure, in most scenarios, after 10 episodes, the consistency between the cloud-side and edge-side results reaches 100%, while in some scenarios it approaches 100%. This phenomenon may be attributed to the exploration-exploitation trade-off in the DQN learning strategy. In reinforcement learning, the \(\varepsilon\)-greedy strategy typically retains a certain probability of exploration, which introduces slight fluctuations or non-zero rewards during the training process. This phenomenon reflects the robustness of the strategy--it maintains the effectiveness of the migration while preserving the generalization ability of the policy.