# Holistic Utility Satisfaction in Cloud Data Centre Network Using Reinforcement Learning
## 5 Simulation Results
### Performance Comparison with Similar Approaches  
In this part, the performance of the proposed reinforcement-learning-based rate allocation algorithm is compared with the dynamic bin-packing resource allocation method in [28] ([PERSON] et al.), the CCEA method in [3], and the game-theory-based resource allocation method ([PERSON] et al.) [48] from the simultaneous cloud user/cloud provider utility satisfaction perspective. A joint and exponentially weighted average--which incorporates all cloud  
Figure 8: Convergence behaviour of average CP utility (7) for multiple user populations (small (blue), medium (red), large (green)).  
Figure 10: Convergence behaviour of average queue function for multiple user populations (small (blue), medium (red), large (green)).  
Figure 9: Convergence behaviour of average reward function for multiple user populations (small (blue), medium (red), large (green)).  
users' and cloud provider utilities into account--utility function was designed for different slots \(t\), as described by Equation (15):  
\[\mathcal{U}(t)\overset{\Delta}{=}\alpha(t)\sum_{i=1}^{M(t)}\mathcal{U}^{(i)}(t)+( 1-\alpha(t))\mathcal{U}^{(CP)}(t) \tag{15}\]  
We previously described the cloud user and cloud provider utility functions in Equations (2) and (7) of Section 3, respectively. The positive \(0<\alpha(t)<1\) is in fact an exponentially weighted parameter to leverage the total cloud user utilities and cloud provider utility at each time frame \(t\). We selected five different cloud data centres (\(N(t)=5\)) that serve 120 demanding cloud users (\(M(t)=120\)) in each time frame \(t\). We repeated each run of the methods 200 different times and, then, took the average result to cover the probabilistic properties of multiple algorithms. The results are presented in Figure 11. It can be verified that the proposed RL-based method improved the mean convergence performance to the optimal solution from the perspective of simultaneous cloud provider/user satisfaction. The most important reason that similar work in [48] and [28] had worse mean satisfaction performance is the fact that these methods do not consider the joint SLA satisfaction for both cloud users and cloud providers and their main point of interest was optimising the resources from only the cloud provider or cloud user point of view. In comparison with our previous work [3], the proposed QLRA algorithm has better convergence behaviour and is able to converge to an optimal resource assignment strategy in the early stages of the simulation.  
In Table 4, we compare the computational complexities of the different algorithms in terms of the mean execution time. As can be verified, the online feature of the proposed algorithm results in a reduced computational time in comparison with similar approaches. On the other hand, we compared the mean holistic utility according to Equation (15) between different methods during 1 time slot and 10 time slots, respectively. It can be verified that, although in the short term, the proposed QLRA method has a bit lower mean value in comparison with the CCEA method, in the long term, it has a comparable performance with it and a superior performance in comparison with the other methods.  
Figure 11: Average joint users’/provider utility comparison.