## 3 Solving Method
### Training Process  
The proposed ADN was trained using the SAC algorithm framework [49], which is an off-policy algorithm based on the maximum entropy actor-critic framework. In this framework, the actor aims to maximize entropy to maintain sufficient exploration. To avoid drastic alterations during the training process, a smooth soft target update rule was  
Figure 4: Architecture of the proposed ADN. The architecture is composed of five components: Input, Encoding, Attention, Fully Connected, and Output. The inputs are encoded as the inputs of attention layers and then operated by the fully connected layers. The probability distribution of task selection is obtained.  
used to progressively update the network parameters. Moreover, to improve stability, SAC adopts a dual Q-network structure, which consists of two independent Q-networks used to calculate the Q-values and uses the minimum value as the target Q-value. The loss function of SAC is defined as follows:  
\[J_{Q}(\Phi)=\mathrm{E}\Big{[}\big{(}Q(S_{t},A_{t})-r(S_{t},A_{t})-\gamma\mathrm{ E}_{S_{t+1}}\big{[}V_{\bar{\Phi}}(S_{t+1})\big{]}\big{)}^{2}\Big{]}, \tag{16}\]  
\[V_{\bar{\Phi}}(s)=\mathrm{E}_{\pi_{\theta}}\big{[}Q_{\bar{\Phi}}(s,a)-\alpha \log\pi_{\theta}(a|s)\big{]}, \tag{17}\]  
\[J_{\pi}(\theta)=\mathrm{E}_{s\sim D}[\mathrm{E}_{a\sim\pi_{\theta}}[\alpha \log\pi_{\theta}(a|s)-Q_{\Phi}(s,a)]], \tag{18}\]  
\[Q_{\Phi}(s,a)=\min\big{(}Q_{\Phi_{1}}(s,a),Q_{\Phi_{2}}(s,a)\big{)}, \tag{19}\]  
where \(Q_{\Phi}(s,a)\) represents the \(Q\) value function, \(\Phi\) is the parameter of the \(Q\) value function, \(r\) is the reward, \(\widetilde{\Phi}\) is the parameter of the target network, and \(Q_{\widetilde{\Phi}}\) represents the target \(Q\) value function whose parameter \(\widetilde{\Phi}\) is the exponential moving average of \(\Phi\). \(\pi_{\theta}\) is the policy network parameterized by \(\theta\). \(\alpha\) is the regularization parameter of entropy. \(D\) is the experience replay pool.  
The structure of the critic network is shown in Figure 5. The training process is illustrated in Algorithm 1.  
```
Input: Initial ADN parameters \(\theta\), \(Q\) value function parameters \(\Phi_{1}\), \(\Phi_{2}\), empty replay buffer \(D\), target network parameters \(\widetilde{\Phi}_{1}\leftarrow\Phi_{1}\), \(\widetilde{\Phi}_{2}\leftarrow\Phi_{2}\) Output: Trained ADN model
1:for number of episode = 1, 2,..., M\({}_{\mathrm{e}}\)do
2: Initial state \(S_{t+\alpha}\):
3:\(done\) = False, \(done\) is a Boolean indicating whether \(S_{t+1}\) is terminal
4:while not done do
5: Observe state \(S_{t}\) and get an action \(a\) from policy \(\pi_{\theta}\)
6: Execute \(a\) and observe next state \(S_{t+1}\)
7: Get reward \(r\) and \(done\)
8: Store (\(S_{t}\), \(a\), \(r\), \(S_{t+1}\), \(done\)) in \(D\)
9: St \(t_{s}\leftarrow S_{t+1}\)
10: if size(\(D\)) - learning size do
11: Randomly sample a batch of (\(S_{t}\), \(a\), \(r\), \(S_{t+1}\), \(d\)) from \(D\)
12: Compute the loss as in (16)-(19)
13: Update \(\theta\), \(\Phi_{1}\), \(\Phi_{2}\) using Adam
14:\(\widetilde{\Phi}_{1}\leftarrow\tau\widehat{\Phi}_{1}+(1-\tau)\Phi_{1}\)
15:\(\widetilde{\Phi}_{2}\leftarrow\tau\widehat{\Phi}_{2}+(1-\tau)\Phi_{2}\)
16:end if
17:end for
```  
**Algorithm 1** Training process based on SCA