## 3 Preliminaries: Network Model and Problem Definition
### Task-Oriented Markov Decision-Making Process  
This paper proposes a task-oriented MDP model for MEC networks, which has an indefinite length of time steps and multiple time steps are performed simultaneously. As shown in Figure 6, the \(x^{i}_{t_{1}}\) time step corresponding to the task starts at the time of its generation \(t_{1}\) and ends at the time of \(x^{i}_{t_{1}}\) completion of processing \(t_{1}+T^{\text{total}}_{i,t_{1}}\). \(x^{i}_{t_{1}}\) After a brief delay, the intelligent agent \(t^{\prime}_{1}\) observes the state at all times \(s(x^{i}_{t_{1}})\) and performs an offloading action on the task \(a(x^{i}_{t_{1}})\). When \(x^{i}_{t_{1}}\) is processed completing, reward feedback for environmental is \(r(x^{i}_{t_{1}})\). Multiple tasks are processed in parallel.  
The TOMDP model includes elements such as state, action, and reward, defined as follows:  
1. Status  
The state information is composed of the observations of the node itself and the observations of all nodes it can connect to. Assuming that the node \(en_{j}\) receives the \(x^{i}_{t}\) summary information of the task at a certain moment, define the relevant \(x^{i}_{t}\) state as \(s(x^{i}_{t})\). We have the following.  
\[s(x^{i}_{t})\triangleq\left[\frac{\lambda_{i,t}}{\mu_{2}},\frac{\tau_{i,t}}{ \mu_{3}},\mathbf{TP}_{i,t},\mathbf{H}_{i},\frac{q^{\text{cache}}_{i}+q^{\text{ comp}}_{i}+q^{\text{tran}}_{i}}{R^{\text{ud}}},\frac{q^{\text{en-comp}}_{i}+q^{ \text{en-ran}}_{i}}{R^{\text{en}}},\frac{j}{n},\mathbf{Link}_{j},th_{min},th_{max}\right] \tag{14}\]  
where \(\lambda_{i,t}\) represents the amount of input data for \(x^{i}_{t}\). \(\tau_{i,t}\) represents the maximum tolerance for delay for \(x^{i}_{t}\). Constants \(\mu_{2}\) and \(\mu_{3}\) are scaling factors used to achieve normalization effects. Vector \(\mathbf{TP}_{i,t}\) represents the type of task. The matrix \(\mathbf{H}_{i}\) represents the \(en_{j}\) user coordinate records stored in. \(j\) represents the objective function of strategy learning. The vector \(\mathbf{Link}_{j}\) represents the \(en_{j}\) connectivity with other nodes. The capacities of the \(ud_{i}\) cache area, calculation queue, transmission queue, and total cache are \(q^{\text{cache}}_{i}\), \(q^{\text{comp}}_{i}\), \(q^{\text{tran}}_{i}\), and \(R^{\text{nd}}\), respectively. The capacities \(en_{j}\) of the calculation queue, transmission queue, and total cache are \(q^{\text{en-comp}}_{j}\), \(q^{\text{en-ran}}_{j}\), and \(R^{\text{en}}\), respectively. The minimum and maximum thresholds for the queue are \(th_{min}\) and \(th_{max}\). Define the \"completion state\" observed by \(en_{j}\) when \(x^{i}_{t_{1}}\) finishes processing as \(s_{c}(x^{i}_{t})\).
2. Action  
The decision vector includes the offloading target selection of the task \(\mathbf{D}_{i,t}\), computing power allocation weight \(ff^{j}_{i,t}\), queue priority weight \(qf_{i,t}\), bandwidth allocation weight \(bf_{i,j}\)  
Figure 6: Task-oriented [PERSON] decision process.  
and thresholds \(th_{min}\) and \(th_{max}\) in the queue. Assuming the \(x_{t}^{i}\) offloading goal of the task is \(et_{j}\), define the \(x_{t}^{i}\) action related to it as \(a\left(x_{t}^{i}\right)\). Specifically, it is expressed as follows: \[a(x_{t}^{i})\triangleq\left[\mathbf{D}_{i,t},ff_{i,t}^{i},qf_{i,t},bf_{i,j},th_{ min},th_{max}\right]\] (15) 3. Rewards The definition of rewards \(x_{t}^{i}\) in this paper is as follows: \[R(x_{t}^{i})\triangleq 1-\frac{Z(x_{t}^{i})}{\frac{\sum_{f_{i}\in\mathrm{Task }}Z(x_{t}^{i})}{\sum_{f_{i}\in\mathrm{Task}}1}}+\frac{1}{2}-PEN(x_{t}^{i})\] (16) where \(\frac{\sum_{f_{i}\in\mathrm{Task}}Z\left(x_{t}^{i}\right)}{\sum_{f_{i}\in \mathrm{Task}}1}\) represents the \(ud_{i}\) average cost of all generated tasks. \(PEN\left(x_{t}^{i}\right)\in\left\{0,1\right\}\) indicates the penalty value of \(x_{t}^{i}\) given by the decision constraint module for the initial decision.