# Holistic Utility Satisfaction in Cloud Data Centre Network Using Reinforcement Learning
## 3 Modelling of the CDCN
### The Utility Function Model for Cloud Users  
Let us assume that a specific cloud customer \(i\) is running a specific application at time frame \(t\), and the successful and quality-guaranteed execution of its application requires in total \(\mathcal{C}^{(i)}(t)\) CPU core, \(\mathcal{D}^{(i)}(t)\) RAM blocks, \(\mathcal{G}^{(i)}(t)\) storage capacity, and connection bandwidth \(\mathcal{B}^{(i)}(t)\) from the CP's resource scheduler. The CPU, storage capacity, RAM level, and connection bandwidth dedicated by \(DC_{j}\) to cloud customer \(i\) at time \(t\) are denoted by \(c_{j}^{(i)}(t)\geq 0,g_{j}^{(i)}(t)\geq 0\), \(d_{j}^{(i)}(t)\geq 0\), and \(b_{j}^{(i)}(t)\geq 0\), respectively. Therefore, according to Equation (1), we have \(\forall i,t\): \begin{table}
\begin{tabular}{c c} \hline \hline
**Sets and Indices** & **Description** \\ \hline \(\mathcal{S}(t)\) and \(\mathcal{Z}^{\ell}(t)\) & \begin{tabular}{c} State and action spaces for \\ time slot \(t\) and iteration \(\ell\) \\ \end{tabular} \\ \hline \(i,j\) & \
\begin{tabular}{c} Indices of cloud users and cloud \\ data centres \\ \end{tabular} \\ \hline \(\ell,L\) &
\begin{tabular}{c} Iteration number and number of \\ training episodes in Q-learning \\ \end{tabular} \\ \hline \hline
**System Parameters** & **Meaning** \\ \hline \(t\) and \(\tau\) & time slot/frame parameter and time frame \\  & period \\ \hline \(\Gamma,\,\gamma,\,\delta,\,\zeta,\,k,\,\alpha\) & Some positive constants \\ \hline
**Variables** & **Description** \\ \hline \(N(t),M(t)\) & \begin{tabular}{c} Number of CDCs and cloud \\ users for time \(t\) \\ \end{tabular} \\ \hline \(\mathcal{C}^{(i)}(t)\),\(\mathcal{D}^{(i)}(t)\), \(\mathcal{G}^{(i)}(t)\),\(\mathcal{B}^{(i)}(t)\) & \begin{tabular}{c} Total assigned CDCN resources \\ to user \(i\) at \(t\) \\ \end{tabular} \\ \hline \(c_{j}^{(i)}(t)\),\(d_{j}^{(i)}(t)\), \(g_{j}^{(i)}(t)\),\(b_{j}^{(i)}(t)\) & \begin{tabular}{c} assigned CDC \(j\) resources \\ assigned to user \(i\) at \(t\) \\ \end{tabular} \\ \hline \(w_{1}(t)\),\(w_{2}(t)\), \(w_{3}(t)\),\(w_{4}(t)\) & Relative resource prices at time \(t\) \\ \hline \(w_{3}(t)\),\(w_{4}(t)\) & \begin{tabular}{c} Unit time per unit resource price of each \\ storage and bandwidth unit at time \(t\) \\ \end{tabular} \\ \hline \(\xi_{j}(t)\) & \
\begin{tabular}{c} Energy dissipation price for \\ CDC \(j\) at time \(t\) \\ \end{tabular} \\ \hline \(\beta_{1}(t)\),\(\beta_{2}(t)\),\(\beta_{3}(t)\) &
\begin{tabular}{c} Positive cloud energy \\ consumption parameters at time slot \(t\) \\ \end{tabular} \\ \hline \(\mathcal{U}^{(i)}(t)\) & Cloud user/customer utility function \\ \hline \(\mathcal{U}^{(CP)}(t)\) & Utility of cloud provider at slot \(t\) \\ \hline \(\mathcal{C}^{CP}(t)\),\(D^{CP}(t)\), \(G^{CP}(t)\),\(B^{CP}(t)\) & Existing CP resource pool at time \(t\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: System parameters. \[\mathcal{C}^{(i)}(t)=\sum_{j=1}^{N(t)}c_{j}^{(i)}(t),\qquad\mathcal{D}^{(i)}(t) =\sum_{j=1}^{N(t)}d_{j}^{(i)}(t),\]  
\[\mathcal{G}^{(i)}(t)=\sum_{j=1}^{N(t)}g_{j}^{(i)}(t),\qquad\mathcal{B}^{(i)}(t)= \sum_{j=1}^{N(t)}b_{j}^{(i)}(t) \tag{1}\]  
Any application type of cloud user \(i\) (such as FTP, video streaming, gaming, etc.) must be quality-guaranteed with an SLA, which is provided by the cloud manager/provider. It is assumed that cloud resource assignment is elastic. The experienced utility/satisfaction level of user \(i\) is a function of its assigned CDCN VMs including RAM, storage capacity, CPU, and also, connection bandwidth, total expected packet loss, and worst-case expected round-trip time. (In some scenarios, such as cloud gaming, the rendered video in the cloud must be delivered to the user by guaranteeing some packet loss ratio and delay thresholds. If these thresholds are not satisfied, the user's perceived quality of experience (QoE) can be substantially deteriorated.) For high-quality delivery of each network application, a minimum level of cloud data centre resources must be guaranteed.