## 3 Proposed DRL-Based Optimization Algorithm
According to \(\mathrm{P}_{3,km}\), if we set \(\psi_{2}=0\), the optimal solutions for \(\mu_{k,m,t}\) is to set \(\mu_{k,m,t}=0,k\in K,m\in\mathcal{M},t\in T\), which are pointless solutions. Hence, the conditions \(\psi_{1}>0\) and \(\psi_{2}>0\) must be satisfied simultaneously when \(\omega_{1}=1\). If the problem \(\mathrm{P}_{3,km}\) has feasible solutions, the problem \(\mathrm{P}_{3,km}\) is transformed into \(\mathrm{P}_{4,km}\), which is given as  
\[\mathrm{P}_{4,km}: \min_{\left\{\eta_{k,m,t}\right\},\left\{f_{m,k,t}\right\},\left\{p_{ k,m,t}\right\}}\left[E_{m,k}^{0}[t]+E_{m,k}^{c}[t]\right] \tag{9a}\] \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]0  
Problem \(\mathrm{P}_{4,km}\) is still non-convex and intractable due to the complex coupling among the variables \(\tau_{k,m,t}\), \(f_{m,k,t}\), and \(p_{k,m,t}\). To address this issue, we adopt block coordinatedescending (BCD) algorithm to optimize \(\tau_{k,m,t}\), \(f_{k,m,t}\), and \(p_{k,m,t}\) alternately. For any given feasible \(\tau_{k,m,t}\), the optimization problem \(\mathrm{P}_{4,km}\) is transformed into \(\mathrm{P}_{5,km}\), which is given as  
\[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]1 \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]2  
The above optimization problem can be further decomposed into the following two manageable sub-problems, namely,  
\[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]3 \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]4 (11b) \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]5 (12a) \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]6  
**Theorem 1**. : _For a given \(\tau_{k,m,t}\), the optimal \(p_{k,m,t}\) and \(f_{k,m,t}\) can be given as_  
\[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]7 \[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]8  
_respectively._  
Proof. : It is easy to prove that problem \(\mathrm{P}_{5,km,1}\) and \(\mathrm{P}_{5,km,2}\) are both convex optimization problem and can be efficiently solved by using the Karush-Kuhn-Tucker (KKT) conditions [30]. Substituting the above results into \(\mathrm{P}_{5,km}\), we have  
\[\left\{\eta_{k,m,t}\right\},\left\{\eta_{k,m,t}\right\},\left\{ \eta_{k,m,t}\right\}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0}\sum_{k=1}^{K}\mu_ {k,0,t}\]9 \[+\omega_{1}\bigg{[}\psi_{1}\sum_{m=0}^{M}\sum_{k=1}^{K}\mu_{k,m,t }\left(E_{m,k}^{0}[t]+E_{m,k}^{c}[t]\right)+\psi_{2}\sum_{m=0}^{M}\sum_{k=1}^{ K}\mu_{k,m,t}\Gamma_{k,t}l_{k,t}(\beta_{k,t})\bigg{]}\Bigg{\}} \tag{6a}\]0  
where  
\[+\omega_{1}\bigg{[}\psi_{1}\sum_{m=0}^{M}\sum_{k=1}^{K}\mu_{k,m,t }\left(E_{m,k}^{0}[t]+E_{m,k}^{c}[t]\right)+\psi_{2}\sum_{m=0}^{M}\sum_{k=1}^{ K}\mu_{k,m,t}\Gamma_{k,t}l_{k,t}(\beta_{k,t})\bigg{]}\Bigg{\}} \tag{6a}\]1 \[+\omega_{1}\bigg{[}\psi_{1}\sum_{m=0}^{M}\sum_{k=1}^{K}\mu_{k,m,t }\left(E_{m,k}^{0}[t]+E_{m,k}^{c}[t]\right)+\psi_{2}\sum_{m=0}^{M}\sum_{k=1}^{ K}\mu_{k,m,t}\Gamma_{k,t}l_{k,t}(\beta_{k,t})\bigg{]}\Bigg{\}} \tag{6a}\]2  
According to Theorem 1, the optimal solution of the problem \(\mathrm{P}_{5,km}\) have closed-form optimal solutions, which are determined by the value of \(\tau_{m,k,t}\).