## 2 System Model
### Problem Formulation  
If the computing pressure of the PMS is too high, i.e., if the VSO allocates too many tasks to the PMS, the PMS may have higher probability for crashing. As the PMS has much stronger computing power than the SMSs, the crashing of the PMS has serious impact on the reliability of the MEC system. Meanwhile, the energy consumption and the task completion rate are also very important to the MEC system. Therefore, in this paper, we aim to minimize the weighted sum of the computing pressure of the PMS, the sum energy consumption of the MEC servers and the IoT devices, and the task dropping cost. The corresponding optimization problem is formulated as \[\mathrm{P}_{1}:\min_{\{\mu_{k,m,t}\},\{\eta_{k,m,t}\},\{f_{m,k,t}\}, \{p_{k,m,t}\}}E\left[\sum\limits_{t=1}^{T}\gamma^{t-1}\Bigg{\{}\omega_{0}\psi_{0} \psi_{0}\sum\limits_{k=1}^{K}\mu_{k,0,t}\right.\] \[\left.+\omega_{1}\Bigg{[}\psi_{1}\sum\limits_{m=0}^{M}\sum\limits _{k=1}^{K}\mu_{k,m,t}\Big{(}E_{m,k}^{0}[t]+E_{m,k}^{c}[t]\Big{)}+\psi_{2}\sum \limits_{m=0}^{M}\sum\limits_{k=1}^{K}\mu_{k,m,t}\Gamma_{k,t}I_{k,t}(\beta_{k, t})\Bigg{]}\right\}\Bigg{]} \tag{4a}\] \[s.t. \sum\limits_{m=1}^{M}\mu_{k,m,t}=1,\mu_{k,m,t}\in\{0,1\},k\in \mathcal{K},m\in\mathcal{M},t\in\mathcal{T},\] (4b) \[\tau_{k,m,t}R_{k,m}[t]\geq\mu_{k,m,t}l_{k,t},k\in\mathcal{K},m\in \mathcal{M},t\in\mathcal{T},\] (4c) \[\frac{\mu_{k,m,t}l_{k,t}c_{k,t}}{f_{m,k,t}}\leq\Delta-\tau_{k,m,t },k\in\mathcal{K},m\in\mathcal{M},t\in\mathcal{T},\] (4d) \[p_{k,m,t}\geq 0,\,p_{k,m,t}\leq p_{k,\max},k\in\mathcal{K},m\in \mathcal{M},t\in\mathcal{T},\] (4e) \[f_{m,k,t}\geq 0,f_{m,k,t}\leq\frac{f_{m,\max}}{\sum\limits_{k=1}^{ K}\mu_{k,m,t}},k\in\mathcal{K},m\in\mathcal{M},t\in\mathcal{T}\] (4f) \[0\leq\tau_{k,m,t}\leq\Delta,k\in\mathcal{K},m\in\mathcal{M},t\in \mathcal{T},\] (4g) \[\omega_{i}\in\{0,1\},i=0,1,k\in\mathcal{K},m\in\mathcal{M},t\in \mathcal{T}, \tag{4h}\]  
where the first term of the objective function represents the computing pressure of the PMS, the second term represents the sum of energy consumption of the network and the task dropping cost; \(\omega_{0}\) and \(\omega_{1}\) are the weights of the two terms above, respectively; \(\omega_{i}=0,i=0,1\) means the corresponding objective is not considered; \(\omega_{i}=1,i=0,1\) means the corresponding objective is considered; \(\psi_{i}>0,i=0,1,2\) is the normalization factors to normalize each term. \(\gamma\in[0,1]\) is the discount factor, which denotes the difference on importance between the future rewards and the present reward [28]; \(p_{k,\max}\) and \(f_{m,\max}\) denote the maximal transmit power of the IoT device \(k\) and the maximal available CPU frequency of the MEC server \(m\), respectively. \(\Gamma_{k,t}>0\) is task dropping cost of the IoT device \(k\) at time-slot \(t\). \(I_{k,t}\) is the indicator function, which is given as  
\[I_{k,t}=\left\{\begin{array}{ll}0,&\beta_{k,t}\;\;\text{is completed}\\ 1,&\beta_{k,t}\;\;\text{is dropped}.\end{array}\right. \tag{5}\]  
(4b) is the offloading decision variable constraint, which guarantees each IoT device's task has been allocated to a MEC server. (4c) and (4d) are the IoT devices' computation tasks constraints to make sure that each IoT device's task can be offloaded and completed. (4e) is the transmit power constraint of the IoT devices. (4f) is the CPU frequency constraint, we assume that IoT devices equally share the CPU at the MEC server \(m\), \(m\in\mathcal{M}\). (4g) and (4h) are the constraints of task offloading time and weights, respectively.  
Due to the binary variable \(\mu_{k,m,t}\) and high coupling constraints, problem \(\mathrm{P}_{1}\) is a non-convex MINLP problem. Furthermore, the computation tasks of the IoT devices and the channel gains are randomly varying during \(T\) consecutive time-slots. Hence, it is impossible to solve the problem at the beginning of the \(T\) consecutive time-slots. Thus, traditional optimization-based methods are not suitable to solve the problem \(\mathrm{P}_{1}\).