## 4 Design of Proposed Algorithm
### MDP Modeling  
For the different types of tasks in this chapter, we model them as multiple MDPs; each MDP corresponds to an individual task in meta-learning, and the aim of each task is to learn an effective offloading strategy for each MDP. We define the task distribution as \(\rho(T)\), and the MDP for each task \(T_{i}\sim\rho(T)\) is denoted as \(T_{i}~{}=~{}(S,A,P,P_{0},R,\gamma)\). To accommodate  
Figure 2: System architecture of MTD3 CO task offloading. different task types, we divide the learning process into two parts: the first, in which each MDP learns its own specific strategy based on the meta-strategy; the second, in which all MDPs' specific policies are extracted into a common meta-strategy and updated for the next learning loop. According to the system model, the status, action, and reward functions of the MDPs are defined below. (1) State Space  
When offloading a subtask, the subtask's execution latency is dependent on the CPU cycles required by the task, the size of the data being uploaded and downloaded, the DAG topology, the decision of the previous task, the transfer rate, the device power, and the MEC resources. The state space of subtask \(t_{i}\) is denoted as \(s_{i}~{}=~{}\left(G,A_{1:i-1},data_{i}^{u},C_{i},data_{i}^{d},T_{i}^{max},B_{ i-1}^{remain}\right)\), where G denotes the DAG topology of the task to which the subtask belongs. \(A_{1:i-1}~{}=~{}\{a_{1},a_{n}~{}\ldots a_{i-1}\}\) is the offload decision record of the task \(t_{i}\) predecessor; \(data_{i}^{u}\), \(C_{i}\), and \(data_{i}^{d}\) are the amount of data uploaded by task \(t_{i}\), the amount of computation of the task, and the amount of data downloaded, respectively; \(T_{i}^{max}\) is the maximum tolerated delay of the task, and exceeding it is defined as task failure; \(B_{i-1}^{remain}\) is the remaining power on the device before executing task \(t_{i}\). To convert the set of subtasks represented by G into a sequence of subtasks and preserve the priority relationship, we add indices to the tasks using \(rank(t_{i})\), and then sort the tasks according to the indices. We define \(rank(t_{i})\) as follows:  
\[rank(t_{i})~{}=~{}\left\{ \begin{array}{ll}T_{i}^{EDGE}&\text{if }t_{i}\in K\\ T_{i}^{EDGE}+\underset{t_{i}\in child(t_{i})}{max}\big{(}rank(t_{i})\big{)}& \text{if }t_{i}\
otin K\end{array} \right., \tag{11}\]  
where \(T_{i}^{EDGE}\) is the time interval between the start of offloading of task \(t_{i}\) and the result obtained from the edge server, and \(child(t_{i})\) is the set of direct subtasks of task \(t_{i}\). According to \(\text{rank}(t_{i})\), \(G\) is converted to a direct parent task index vector and a direct child task index vector of task \(t_{i}\), and the size of this vector is set to the maximum value contained in all applications or filled with 0 if it is less than the maximum value. (2) Action Space  
The offload strategy for each task includes the offload decision and _WPT_ service time, \(a_{i}~{}=~{}\left(d_{i},T_{i}^{WPT}\right)\), where \(d_{i}\) denotes the offload decision (\(d_{i}~{}=~{}\{0,1\}\), where 0 means local execution and 1 means offload execution), and \(T_{i}^{WPT}\) represents the duration of _WPT_ service before each task execution. Here, it is specified that \(T_{i}^{WPT}~{}=~{}t\). To simulate discrete actions, \(t\) is taken as a multiple of 0.01. _WPT_ service duration and offload decisions affect task latency and device energy consumption together. (3) Reward Function  
The optimal task scheduling decision should have the lowest task processing latency and the lowest energy consumption for mobile devices. If a task fails to execute due to inadequate power or execution timeouts, it is called a \"task failure\", which is defined as \(T_{i}(fail)\). The power consumed by each task \(t_{i}\) execution may be expressed as the difference of total energy consumption \(\Delta E_{i}~{}=~{}E_{A_{1:i}}^{total}-E_{A_{1:i-1}}^{total}\), and task latency can be expressed as \(\Delta T_{i}~{}=~{}E_{A_{1:i}}^{all}-E_{A_{1:i-1}}^{all}\).