## 4 SACDTO Strategy
### SACDTO Implementation
Therefore, if you want to use the scheme proposed in this paper to complete the offloading decision of applications, but these applications do not have relevant topological information in the real data set, you can use the synthetic DAG generator [31] to generate DAGs with various topologies to represent the dependencies between heterogeneous applications. The generator can adjust the aspect ratio (the aspect ratio determines the width of the DAG) and the Communication Calculation Ratio (CCR), and the changes in these two values will construct different topologies. In the simulation experiment, according to the different distance between the terminal and the MEC host, there will be different channel gains and different transmission rates. Assuming that this is ignored, the default transmission rate is set to 1 Mbps in the experiment. Set the CPU clock frequency \(f_{i}^{loc}\) of the local device to 1 GHz and the CPU clock frequency \(f_{m}^{s}\) of the MEC server to 8 GHz. The sending power \(P_{send}\) and receiving power \(P_{rec}\) are 1.26 W and 1.2 W, respectively. The DAG task configuration file size \(d_{i}\) is set to 5 KB to 50 KB, and the number of clock cycles required by a single task is set to \(10^{7}\) to \(10^{8}\) cycles/sec. In each agent, the actor and the critics consist of an input layer, two hidden fully-connected layers, and an output layer. Both hidden fully connected layers are set to 256 neurons. We summarize the hyperparameters of SACDTO implementation in Table 3.

Figure 2 compares the convergence curves of the average rewards of SACDTO, PPOTO, D3 QNTO, and RTO. Since the reward value function takes the local calculation strategy as the benchmark comparison, the convergence of all tasks executed locally can be ignored. By default, we assume that the number of MEC servers is three and that we have 30 tasks to process. Obviously, as the number of episodes increases, the average reward value of the three schemes based on DRL increases until they converge. However, the convergence curve of RCO has been maintained at a low level, and there is no upward trend, which indicates that RTO cannot learn self-adaptive offloading action. D3 QNTO converged gradually after the 150 th round, and PPOTO converged at the 20 th round. Although the PPOTO converged faster, the average reward of this strategy was less than D3 QNTO by 5.74%. SACDTO strategy gradually converges after 125 rounds, and its result has 11.21% more reward than PPOTO, which is the best result among the above algorithms. Since SAC can make use of all useful behaviors in each state and explore more useful behavior space through random strategy and maximum entropy target, the proposed SACDTO can obtain a higher average reward value compared with PPOTO and D3 QNTO. This reflects that the SAC-based offloading scheme can learn more appropriate offloading strategies than PPO and D3 QN. Figure 3 shows the Time and Energy Reduced Scale (TERS) of the four offloading schemes when the number of tasks increases. TERS refers to the reduction ratio between the total cost calculated by the corresponding weight coefficient after unloading and the total cost consumed by the local task execution. Since it is a trade-off between delay and energy consumption, this value can also equal to QoS. Since TERS are also calculated against the local execution policy, ALOC can be ignored. By default, the aspect ratio of DAG is set to 0.45, the number of MEC servers is set to 3, and CCR is set to 0.5 (the smaller the value, the more intensive the computation). Compared with PPOTO, D3 QNTO, and RTO, SACDTO has an average increment of 2.4%, 54.82%, and 79.28%, respectively, which indicates that SACDTO has the highest proportion of optimization compared with the local execution strategy. This is because with the increase in the number of tasks, the delay and