# Resource Allocation and Offloading Strategy for UAV-Assisted LEO Satellite Edge Computing
## 4 Algorithm Design
### Task and Computing Resource Allocation Strategy
The actor network of the agent is responsible for generating actions and contains two components: the actor network \(\mu^{\theta}\) and \(\mu^{\theta^{\prime}}\), where \(\theta\) and \(\theta^{\prime}\) are the network parameters. The critic network of agent is responsible for evaluating actions and contains two components: the critic network \(Q^{\phi}\) and \(Q^{\phi^{\prime}}\), where \(\phi\) and \(\phi^{\prime}\) are the network parameters. The DDPG-LSTM algorithm has three main elements. * State The state consists of the IoT device task information and satellite resource information. \[S(t)=\{S_{i},f_{j}\}\] (39)  
Figure 2: Architecture for the DDPG-LSTM-based computation offloading scheme. where \(S_{i}\) denotes the computational task size of the ground-based IoT device and \(f_{j}\) denotes the computational resources of the LEO satellite. * Action. The action is composed of the task allocation vector \(S_{ij}\) and the computational resource allocation vector \(F_{ij}\) to obtain the action space A of the system. \[A(t)=\{S_{i1},S_{i2},\dots,S_{i\
u},f_{j1},f_{j2},\dots,f_{jn}\} \tag{40}\]
* Reward. In this work, a function is introduced to explain the amount of change in the system cost when action \(A_{t}\) is taken in system state \(S_{t}\). It is expressed as  
\[R(t)=U^{t}-U^{t+1} \tag{41}\]  
Here, \(U^{t}\) and \(U^{t+1}\) means the latency and energy consumption at time point t and the next time point. The magnitude of the difference represents the cost reduction achieved by \(A_{t}\); the system benefit of taking action \(A_{t}\) is \(U^{t}\). The MDP is aimed at maximising the Reward Sum expected to be received, and can therefore be formulated as follows  
\[\max\ Y=\sum_{t=0}^{T-1}Y^{t} \tag{42}\]  
The process of DDPG-LSTM is shown in the Algorithm 2, where the weight parameters and the replay buffer are initialised. In each training round, the agent is given a state \(Z_{t}\), decides on an action \(a_{t}\), performs the action and receives a reward \(R_{t}\). Then, the replay buffer stores the experience transformation and the chosen batch size of M. Finally, the networks are refreshed. ```
1:task \(S_{i}\), computing resource \(f_{j}\)
2:task allocation \(S_{ij}\), computing resource \(f_{ji}\)
3:Actor_Critic weight parameters randomised initialisation : \(\omega,\theta,\omega^{\prime},\theta^{\prime}\)
4:Initialising Replay Buffer \(M\), mini-batch size \(B\), train episode threshold \(C\);
5:for\(m=1,Ep\)do
6: Initialize environment;
7: Initialize LSTM states in the network;
8: Reception of primary state \(Z_{t}\);
9:for\(t=1,T\)do
10: Choose \(a_{t}\) by online network, \(a_{t}=H(Z_{t}|w)+n_{o}\);
11: Implementation \(a_{t}\), obtain \(R_{t}\) and switch to the state \(Z_{t+1}\);
12: Store \((Z_{t},a_{t},R_{t},Z_{t+1})\) in \(M\);
13:if\(m\geq C\)then
14: Sample random mini-batch transitions \((Z_{i},a_{i},R_{i},Z_{i+1})\) from \(R\);
15: Using the loss function and policy gradient to refresh online critic and actor parameters;
16: Refresh parameters of the target network;
17:endif
18:endfor
19:endfor
```  
**Algorithm 2** DDPG-LSTM-based task and resource allocation algorithm