# A Novel Deep Reinforcement Learning Approach for Task Offloading in MEC Systems
## 3 System Model
### Computing Models  
(1)Local computingFor the user, \(m\), if the agent decides to process the task, \(u_{m}(t)\), at MD, \(m\), at slot \(t\), then the task will be placed in the user's queuing buffer and be processed at upcoming slots. We use \(\varphi_{m}^{loc}(t)\) to define the amount of time slots that \(u_{m}(t)\) has to wait before being executed and \(L_{m}^{loc}(t)\) to present when the task is totally executed or dropped if it has not been completed at the deadline. For task \(u_{m}(t)\), \(\varphi_{m}^{loc}(t)\) is computed as follows:  
Figure 1: System model.  
\[\varphi_{m}^{loc}(t)=\max_{t^{{}^{\prime}}\in\{0,1,\ldots,t-1\}}L_{m}^{loc}(t^{{}^{ \prime}})-t+1 \tag{2}\]  
Obviously, \(\varphi_{m}^{loc}(t)\) depends on the tasks before task, \(u_{m}(t)\), and, when all of them have been completed or dropped, \(u_{m}(t)\) will be processed at \(t+\varphi_{m}^{loc}(t)\). In addition, \(L_{m}^{loc}(t)\) also represents the total computation delay from arriving at the MD, \(m\), to being completed or dropped; it is calculated as:  
\[L_{m}^{loc}(t)=\min\{t+\varphi_{m}^{loc}(t)+\frac{\lambda_{m}(t)}{f_{m}\mu/ \zeta_{m}}-1,t+\pi_{m}-1\}, \tag{3}\]  
where \(f_{m}\) denotes the CPU processing capacity of MD, \(m\), and \(\zeta_{m}\) denotes the CPU cycles MD, \(m\), required for processing a unit data [30].  
_(2) Edge Computing_  
When the task \(u_{m}(t)\) is offloaded to ED \(n\) for processing, we define \(L_{m,n}^{edge}(t)\) to present when the task is executed or dropped by the service. Since the queue dynamic of ED \(n\), the MD, \(m\), is unclear of the figure of \(L_{m,n}^{edge}(t)\) until the task processing is finished. Let \(\hat{L}_{m,n}^{edge}(t)\) represent when the task begins to be processed:  
\[\hat{L}_{m,n}^{edge}(t)=\max\{t,\max_{t^{{}^{\prime}}\in\{0,1,\ldots,t-1\}}L_{m,n}^{edge}(t^{{}^{\prime}})+1\} \tag{4}\]  
The MEC servers are assumed to have adequate computing resources. Thereby, the servers are able to handle various tasks transmitted from various MDs. That is to say, once the tasks arrive at the MEC servers, they will be executed right away, and thus there is no queuing delay.  
_(3) Communication model_  
In this paper, we assume a wireless network that mobile devices communicate on orthogonal channels. Denote \(h_{m,n}\) as the uplink channel gain between the user \(m\) and the ED \(n\). Let \(P\) denote the user's transmission power. The uplink transmission rate (bits per second) in the system to offload \(u_{m}(t)\) is calculated as follows:  
\[r_{m,n}^{trans}=Wlog_{2}(1+\frac{h_{m,n}^{2}P}{\sigma^{2}}), \tag{5}\]  
where \(W\) is the channel bandwidth and \(\sigma^{2}\) is the noise power at the edge server.  
Once the task, \(u_{m}(t)\), is placed in the transmission queue at time slot \(t\), we denote \(\varphi_{m}^{trans}(t)\) to define the number of time slots that \(u_{m}(t)\) will wait for being transmitted; \(L_{m}^{trans}(t)\) represents when the task is totally transmitted or dropped if it has not been transmitted at the deadline. For task \(u_{m}(t)\), \(\varphi_{m}^{trans}(t)\) is computed as follows:  
\[\varphi_{m}^{trans}(t)=\max_{t^{{}^{\prime}}\in\{0,1,\ldots,t-1\}}L_{m}^{trans} (t^{{}^{\prime}})-t+1 \tag{6}\]  
Furthermore, the time \(L_{m}^{trans}(t)\) is computed as follows:  
\[L_{m}^{trans}(t)=\min\{t+\varphi_{m}^{trans}(t)+\frac{\lambda_{m}(t)}{r_{m,n}^ {trans}}-1,t+\pi_{m}-1\} \tag{7}\]  
The offloading process includes task data transfer and its computation result between MDs and EDs. Since the wirelines between the BS and the MEC servers ensure much faster transmission speed than wireless channels, we ignore the transmission delay between them.