# Research on Task-Oriented Computation Offloading Decision in Space-Air-Ground Integrated Network
## 2 Materials and Methods
### DQN-Based Computation Offloading Decision Algorithm
Each task that executes offloading selects the size of the parameter value \(\alpha\) and \(\beta\) according to its characteristic attributes. For example, for military emergency tasks, the parameter value \(\alpha\) can be set bigger. The task computation offloading optimization goals are as follows:  
\[minimize\Bigg{\{}Totalcost(C)=\alpha\sum_{v_{i}\in V}\sum_{i=1}^{m}C_{T_{in}}+ \beta\sum_{v_{i}\in V}\sum_{i=1}^{m}C_{E_{in}}\Bigg{\}} \tag{12}\]  
\[st.\quad\forall:f_{maxgive}\geq w_{in} \tag{13}\]  
\[\alpha+\beta=1 \tag{14}\]  
where \(m\) represents the number of subtasks of the offloading task \(v\) of the terminal device \(n\), \(i\) represents the number of currently executed subtasks \(0\leq i\leq m\), and \(C\) represents the weighted total cost of delay and energy consumption. After obtaining the above-mentioned optimal strategy, it is also necessary to compare its total cost with the total cost of the local execution of the task. If the offloading cost is greater than the local execution cost, the task is executed locally, otherwise, the execution is offloaded. For edge computing nodes, space-based backbone nodes, and ground-based backbone nodes in the Space-Air-Ground Integrated Network, there are differences in the computing storage performance of offloading sites. Therefore, the calculation equations for delay and energy consumption of different offloading sites are given, as shown in Equations (6) to (11), the reward function is shown in Equation (15). \[r_{t}=1/[\alpha T_{t_{i}}+\beta E_{t_{i}}] \tag{15}\]  
Suppose the terminal takes action \(a_{t}\) at time \(t\), the environment transitions from state \(s_{t}\) to \(s_{t+1}\), and the winning case is \(r_{t}=r(s_{t},a_{t},s_{t+1})\);\(s_{t}\) and \(a_{t}\) determine the probability distribution of \(s_{t+1}\) and \(r_{t}\), and \(Q_{t}\) is the action estimate of the initial state action pair at time \(t\). The optimization process of action-value function in the Q-learning process is shown in Equations (16) and (17). \[T_{t_{i}}=FT_{t_{i}}^{ed}=T_{t_{i}}^{ed}+RT_{t_{i}}^{ed}\qquad(offloaded\ to\ edge\ compute\ node) \tag{7}\]0  
\[T_{t_{i}}=FT_{t_{i}}^{ed}=T_{t_{i}}^{ed}+RT_{t_{i}}^{ed}\qquad(offloaded\ to\ edge\ compute\ node) \tag{7}\]1  
where \(\lambda_{t}\) represents the learning rate. The larger the value of \(\lambda_{t}\) is, the faster the convergence speed will be. However, the value should not be too large, otherwise, the convergence will be unstable and the optimal computation offloading decision will be affected. Since Q-learning cannot store all possible states that in the complex Space-Air-Ground Integrated offloading network in a Q table, introducing the neural network to estimate the \(Q(s,a)\) generated in Q-learning can reduce the state space in the process of computation offloading decision and accelerate the convergence rate of the offloading process. Equations (16) and (17) is used to calculate the partial action \(Q(s,a)\) value of a single task during the task offloading decision period \(t\), which is stored as the experience value in the memory bank \(\Omega\) and used as the training data set of the convolutional neural network. Then the vector containing the value of each action \(Q(s,a)\) is output through two fully connected convolutional network layers. The neural network is used to convert a single \(Q\) value into a \(Q\) network, and then the backpropagation and gradient descent methods are used to minimize the loss function, as shown in Equation (18):  
\[T_{t_{i}}=FT_{t_{i}}^{ed}=T_{t_{i}}^{ed}+RT_{t_{i}}^{ed}\qquad(offloaded\ to\ edge\ compute\ node) \tag{7}\]2  
Finally, Equations (8) and (11) choose the optimal offloading decision to execute the task. The DQN computation offloading algorithm uses the deep convolutional neural network to nonlinearly approximate the action-value function in Q-learning during the computation offloading decision process, adopts the experience playback method to train the Q-learning process, and randomly extracts experience data from the state space to learn.