### Properties  
We state the characteristics of our algorithm as Proposition 2.  
Proposition 2: _If the drift function satisfies the drift condition for all time slots and all possible \(Z(t)\) and the expected penalty function \(\mathbb{E}[E(t)]\) is lower bounded by a finite value \(\zeta_{min}\), the long-term time-averaged expected energy a MES consumes when it uses our online service time allocation algorithm satisfies_  
\[\lim_{T\to\infty}\frac{1}{T}\sum_{t=0}^{T-1}\mathbb{E}[E(t)]\leq\zeta_{opt}+ \frac{B}{V^{\prime}}, \tag{29}\]_where \(\zeta_{opt}\) is the optimal time-averaged penalty value and \(B\) a positive constant. In addition, the long-term time-averaged expected virtual queue length satisfies_  
\[\lim_{T\to\infty}\frac{1}{T}\sum_{t=0}^{T-1}\sum_{i\in N}\mathbb{E}[|Z_{i}(t)|] \leq\frac{B+V(\zeta_{opt}-\zeta_{min})}{\epsilon}, \tag{30}\]  
_where \(\epsilon\) is a non-negative constant._  
Proof.: We followed the approach in [39]. Suppose there are constants \(B>0\) and \(\epsilon\geq 0\) such that the following drift condition holds for all time slots and possible \(Z(t)\)s:  
\[\Delta L(Z(t))\leq B-\epsilon\sum_{i\in N}|Z_{i}(t)|. \tag{31}\]  
Let us assume that the expected penalty function \(\mathbb{E}[E(t)]\) is lower bounded by a finite value \(\zeta_{min}\). Then, for all time slots and all possible resource allocation actions, we have  
\[\mathbb{E}[E(t)]\geq\zeta_{min}. \tag{32}\]  
We denote the optimal time-averaged penalty value as \(\zeta_{opt}\). That is, for all time slots and all possible resource allocation actions, we obtain  
\[\mathbb{E}[E(t)]\leq\zeta_{opt}. \tag{33}\]  
Then, for a time slot \(t\), the following inequality holds.  
\[\Delta L(Z(t))+V\mathbb{E}[E(t)]\leq B+V\zeta_{opt}-\epsilon\sum_{i\in N}|Z_{i }(t)|. \tag{34}\]  
Taking expectations on both sides and applying the law of iterated expectations for \(t=0,\ldots,T-1\), we have  
\[\mathbb{E}[L(Z(T))]-\mathbb{E}[L(Z(0)]+V\sum_{t=0}^{T-1}\mathbb{E}[E(t)]\leq T (B+V\zeta_{opt})-\epsilon\sum_{t=0}^{T-1}\sum_{i\in N}|Z_{i}(t)|. \tag{35}\]  
Dividing both side of Equation (35) by \(TV\), we obtain  
\[\frac{\sum_{t=0}^{T-1}\mathbb{E}[E(t)]}{T}\leq\zeta_{opt}+\frac{B}{V}+\frac{ \mathbb{E}[L(Z(0))]-\mathbb{E}[L(Z(T)]}{TV}. \tag{36}\]  
Then, by taking \(T\to\infty\), we obtain  
\[\lim_{T\to\infty}\frac{1}{T}\sum_{t=0}^{T-1}\mathbb{E}[E(t)]\leq\zeta_{opt}+ \frac{B}{V}.\]  
Because \(0\leq V\sum_{t=0}^{T-1}\mathbb{E}[E(t)]\leq\infty\) and \(0\leq L(Z(T))\leq\infty\), by dividing both sides of Equation (35) by \(\epsilon T\) and rearranging terms, we also obtain the following inequality.  
\[\frac{1}{T}\sum_{t=1}^{T-1}|Z_{i}(t)| \leq\frac{B+V\zeta_{opt}}{\epsilon}+\frac{V\sum_{t=0}^{T-1}\mathbb{ E}[E(t)]}{\epsilon T}+\frac{\mathbb{E}[L(Z(0))-\mathbb{E}[L(Z(T))]}{\epsilon T} \tag{37}\] \[\leq\frac{B+V\zeta_{opt}}{\epsilon}+\frac{VT\zeta_{min}}{\epsilon T }+\frac{\mathbb{E}[L(Z(0))]}{\epsilon T}\] (38) \[=\frac{B+V(\zeta_{opt}-\zeta_{min})}{\epsilon}+\frac{\mathbb{E}[L (Z(0))]}{\epsilon T}. \tag{39}\]By taking \(T\to\infty\), we obtain  
\[\lim_{T\to\infty}\frac{1}{T}\sum_{t=1}^{T-1}|Z_{i}(t)|\leq\frac{B+V(\zeta_{opt}- \zeta_{min})}{\epsilon}. \tag{40}\]