# A Multi-Branch DQN-Based Transponder Resource Allocation Approach for Satellite Communications
## 4 Experiments
### Results and Analysis  
Based on the zero-waste and non-zero-waste datasets generated in 4.1.1, we conduct comparative experiments on the resource allocation performance and running time performance, which proves the effectiveness of the proposed MBDQN model. Specifically, the comparative experiments of this paper are shown in Table 5, and the performance of MBDQN and the traditional greedy and heuristic algorithms are mainly compared. In addition, the experimental results and how the reinforcement learning-based MBDQN model affects the algorithm for satellite resource allocation are analyzed. As shown in Table 5, compared with the greedy algorithm and the heuristic algorithm, MBDQN gives rise to a significant improvement in the average resource occupancy performance (AOP) and runtime overhead (RO) when tested on the zero-waste dataset. MBDQN can substantially improve the AOP compared with the greedy algorithm (Bottom Left [5]) within the time limit. In addition, the performance improvement of MBDQN is more obvious with the increase of the satellite task number. For instance, when M is equal to 40, the performance of MBDQN is obviously higher than that of GA-PSO [10] (93.9% vs. 95.1%, AOP). This is due to MBDQN's ability to learn from prior experience, and different branches can accurately learn and extract multi-state features, rather than heuristic exploration for each prediction. In particular, in terms of time complexity, MBDQN significantly reduces the runtime overhead of a complete allocation (32.54 s vs. 1.47 s, \(M=40\)). The reason is that RL-based methods get rid of the iterative process time compared with heuristic algorithms such as Bee Colony [6] and GA-PSO. MBDQN's training, albeit long, can be performed offline, and only the trained model is used for inference in practice. To assess the performance of MBDQN in a more extensive and realistic setting, we also conduct experiments on the NZW-Dataset generation method. The parameters for the training and testing of each model are the same as the previous settings. The experimental results (presented in Table 6) are on par with those on ZW-Dataset. Due to the non-zero waste, the optimal allocation result on this dataset cannot be known, but AOP can still be used to compare the performance of each method. Compared with Bee Colony and GA-PSO, MBDQN also achieves the best performance in terms of both AOP (87.9 vs. 88.4 vs. 89.6, \(M=40\)) and runtime overhead (21.39 vs. 31.81 s vs. 1.52 s, \(M=40\)). From Tables 5 and 6, it can be concluded that: (a) compared with the greedy (i.e., Bottom Left) and heuristic (i.e., Bee Colony, GA-PSO) algorithms, the proposed MBDQN has resulted in boosted AOP on both non-zero waste and zero waste datasets, and the improvement becomes more obvious as the number of tasks increases. (b) The MBDQN model has the advantage of shorter allocation time compared to heuristic algorithms such as Bee Colony and GA-PSO, which is due to the fact that MBDQN does not need iteration in the test phase. The network itself has a relatively small number of layers and the removal of BN further improves its inference speed. These results indicate that the trained multi-branch-based DQN model can effectively solve the problem of satellite resource allocation, \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Task Number**} & \multicolumn{2}{c}{**Bottom Left [6]**} & \multicolumn{2}{c}{**Bee Colony [7]**} & \multicolumn{2}{c}{**GA-PSO [11]**} & \multicolumn{2}{c}{**MBDQN (Ours)**} \\ \cline{2-9}  & **AOP (\%)** & **RO (s)** & **AOP (\%)** & **RO (s)** & **AOP (\%)** & **RO (s)** & **AOP (\%)** & **RO (s)** \\ \hline \(M=10\) & 85.2 & 0.13 & 97.5 & 4.16 & 97.9 & 10.42 & 97.7 & 0.59 \\ \(M=20\) & 86.9 & 0.22 & 93.5 & 9.32 & 95.1 & 18.96 & 95.3 & 0.83 \\ \(M=30\) & 89.1 & 0.30 & 93.0 & 13.05 & 93.7 & 24.50 & 94.6 & 1.21 \\ \(M=40\) & 91.3 & 0.37 & 92.6 & 19.41 & 93.9 & 32.54 & 95.1 & 1.47 \\ \hline \hline \end{tabular}
\end{table}