## 4 SACDTO Strategy
### SACDTO Implementation  
The offloading process consists of the following three steps:  
Step 1: The tasks of DAG are topologically sorted, and all task nodes are sorted according to the descending order of the rank value of the tasks, so as to obtain the task sequence, which can be defined as:  
\[rank(v_{i})\ =\ \left\{ \begin{array}{l}T\_tot_{i},v_{i}\
otin exit\\ \max_{(v_{i},\varphi_{j})\in\epsilon}(rank(v_{j}))\ +\ T\_tot_{i},v_{i}\in exit \end{array} \right. \tag{23}\]  
where \(T\_tot_{i}=\emph{Finish\_T\,i}^{\emph{up}}+\emph{Finish\_T\,i}^{\emph{up}}+ \emph{Finish\_T\,i}^{\emph{up}}\) and _exit_ represents the set of exit task nodes. Step 2: According to the description of the state space in the offloading model in Section 4.2, the tasks are transformed into vector sequences, which are used as the input of the neural network. The output of the neural network is the probability of offloading. Step 3: The offloading action with high probability is taken as the unloading decision \(a_{i}\) of task \(v_{i}\), and the mobile device and the MEC host cooperate to complete task \(v_{i}\) according to the \(a_{i}\). The offloading decision of each task is made by SACDTO, and the detailed process of SACDTO is shown in Algorithm 1. ``` Input: Episode, environment, batch size, replay buffer size
1 Initialize the networks: \(Q_{01}\)\(\rightarrow\)\(R^{\left\lfloor A\right\rfloor}\),\(Q_{02}\)\(\rightarrow\)\(R^{\left\lfloor A\right\rfloor}\),\(\pi_{\theta}\)\(\rightarrow\)\([0,\,1]^{\left\lfloor A\right\rfloor}\)  Initialize the target network: \(\hat{Q}_{\theta 1}\)\(\rightarrow\)\(R^{\left\lfloor A\right\rfloor}\), \(\hat{Q}_{\theta 2}\)\(\rightarrow\)\(R^{\left\lfloor A\right\rfloor}\)  Initialize the target network weights: \(\hat{\theta}_{1}\)\(\leftarrow\)\(\theta_{1}\), \(\hat{\theta}_{2}\)\(\leftarrow\)\(\theta_{2}\)  Initialize the replay buffer: \(D\)\(\leftarrow\)\(\varnothing\)
2 for each episodedo
3 Obtain state \(\mathsf{S}_{t}\) from environment;
4 while not done:
5 Allocate computing resources to each MEC server;
6 Determine the current task to work on based on the priority list;
7\(a_{t}\)\(\rightarrow\)\(\pi_{\theta}\) (\(a_{t}\)\(\mid\)\(s_{t}\));
8 Update the remaining computing resources for each MEC server according to \(a_{t}\);
9\(s_{t+1}\)\(\rightarrow\)\(p\) (\(s_{t+1}\)\(\mid\)\(s_{t+1}\)\(\mid\)\(s_{t+1}\));
10\(D\)\(\leftarrow\)\(D\cup\{(s_{t},a_{t},r(s_{t},a_{t}),s_{t+1})\);
11 ifcurrent episode % learning interval step=\(0\)then
12

## 5 Performance Evaluation

This section gives the experimental results and performance evaluation of SACDTO. We first introduced the meaning of the different parameters in the DAG generator. Secondly, we set the simulation environment and hyperparameters, and give the convergence analysis of SACDTO an average reward. Finally, the performance of this algorithm is analyzed by comparing it with four benchmark algorithms. ### Baseline Approaches

To verify the effectiveness and convergence of the proposed offloading strategy and evaluate the performance of this strategy under different parameters from multiple perspectives, we consider the following three computation offloading schemes for comparison. * Proximal Policy Optimization-based Task Offloading (PPOTO): PPO is an improved algorithm of Policy Gradient, but still has the disadvantage of low sampling efficiency;
* Dueling Double Deep-Q Network-based Task Offloading (D3 QNTO): Combines the advantages of Dueling DQN and Double DQN, and improves the training algorithm and model structure of DQN;
* Random Task Offloading (RTO): Computation tasks are offloaded randomly;
* All Local (ALOC): All computing tasks are executed locally. ### Simulation and Results

In practice, applications can be modeled as DAGs with various topologies, but current real datasets contain information for only a very limited number of applications [30].