## 4 Experimental Results
### Training Results  
To demonstrate the convergence of the proposed ADN, the episode reward for each episode during the training process is illustrated in Figure 6. The training curves of the ADN indicate that the episode reward increases rapidly at the beginning and then oscillates around a certain value. This demonstrates that the decision network can learn effectively from experience and is good at convergence. The training curve oscillates because the exploration strategy during the training process is stochastic. In Figure 6a, the training curve almost converges to a straight line because the stochastic strategy obtains an optimal solution. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Learning rate of the actor & \(1\times 10^{-4}\) \\ Learning rate of the critic & \(1\times 10^{-3}\) \\ Soft update factor \(\tau\) & 0.05 \\ Entropy weight \(\alpha\) & 0.01 \\ Conv-1d1 & in channels = 2, out channels = 256, kernel size = 1, stride = 1 \\ Conv-1d2 & in channels = 3, out channels = 256, kernel size = 1, stride = 1 \\ Conv-1d3 & in channels = 2, out channels = 256, kernel size = 1, stride = 1 \\ Conv-1d4 & in channels = 2, out channels = 256, kernel size = 1, stride = 1 \\ Conv-1d5 & in channels = 3, out channels = 256, kernel size = 1, stride = 1 \\ Linear1 & in features = 768, out features = 256 \\ Linear2 & in features = 256, out features = 1 \\ Linear3 & in features = 512, out features = 256 \\ Linear4 & in features = 256, out features = 1 \\ Linear5 & in features = 512 * number of tasks, out features = 256 \\ Linear6 & in features = 256, out features = number of tasks \\ GRU & input size = 256, hidden size = 256, num layers = 1 \\ Dropout & 0.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont._  
Figure 6: _Cont._The training results of the non-attention decision network (NADN) are also shown in Figure 6. Compared to ADN, NADN only removes the attention layers in the actor, while the rest are the same as in ADN. The training curves show that NADN oscillates around a value after convergence that is always smaller than that of ADN. Moreover, the converged stochastic strategy did not receive the highest episode reward in the training history; that is, without attention layers, the decision network fell into a local optimum. This proves that the designed attention network is beneficial for the training process to converge to better strategies. To show the sensitivity of the parameters of the proposed algorithm, the training results for different values of ten parameters are illustrated in Figure 7. It is worth noting that the optimal solutions for 100-task scenarios can be achieved using a stochastic strategy, and thus, we present the training results of 200-task scenarios (similar conclusions can be drawn in other scenarios). Figure 6: Training performance of ADN and NADN. (**a**) Scene: Number of tasks: 100. (**b**) Scene: Number of tasks: 200. (**c**) Scene: Number of tasks: 300. (**d**) Scene: Number of tasks: 400. (**e**) Scene: Number of tasks: 500. (**f**) Scene: Number of tasks: 600. (**g**) Scene: Number of tasks: 700. (**h**) Scene: Number of tasks: 800. (**i**) Scene: Number of tasks: 900. (**j**) Scene: Number of tasks: 1000. Figure 7: _Cont._  
The results of the parameters set in Table 4 are shown in blue curves. It can be observed from Figure 7 that the memory capacity, batch size, learning size, discount factor, learning rate of the actor, learning rate of the critic, soft update factor, entropy weight, hidden layer dimension of the networks, and dropout are all sensitive parameters for the algorithm. Deviation from the values in Table 4 leads to non-convergence or convergence to smaller episode rewards. The sensitivity of the maximum number of episodes can be found in Figure 6. The training curve oscillates around a value after the training episode reaches a certain number. Table 5 presents the training times for ADN and NADN. For scenarios with different numbers of tasks, the decision networks were trained using 100 episodes.