# Holistic Utility Satisfaction in Cloud Data Centre Network Using Reinforcement Learning
###### Abstract  
Cloud computing leads to efficient resource allocation for network users. In order to achieve efficient allocation, many research activities have been conducted so far. Some researchers focus on classical optimisation theory techniques (such as multi-objective optimisation, evolutionary optimisation, game theory, etc.) to satisfy network providers and network users' service-level agreement (SLA) requirements. Normally, in a cloud data centre network (CDCN), it is difficult to jointly satisfy both the cloud provider and cloud customer' utilities, and this leads to complex combinatorial problems, which are usually NP-hard. Recently, machine learning and artificial intelligence techniques have received much attention from the networking community because of their capability to solve complicated networking problems. In the current work, at first, the holistic utility satisfaction for the cloud data centre provider and customers is formulated as a reinforcement learning (RL) problem with a specific reward function, which is a convex summation of users' utility functions and cloud provider's utility. The user utility functions are modelled as a function of cloud virtualised resources (such as storage, CPU, RAM), connection bandwidth, and also, the network-based expected packet loss and round-trip time factors associated with the cloud users. The cloud provider utility function is modelled as a function of resource prices and energy dissipation costs. Afterwards, a Q-learning implementation of the mentioned RL algorithm is introduced, which is able to converge to the optimal solution in an online and fast manner. The simulation results exhibit the enhanced convergence speed and computational complexity properties of the proposed method in comparison with similar approaches from the joint cloud customer/provider utility satisfaction perspective. To evaluate the scalability property of the proposed method, the results are also repeated for different cloud user population scenarios (small, medium, and large). Keywords:CDCN; QoS; VM; reinforcement learning; resource assignment +
Footnote †: journal:  
On the other side, the cloud provider also has its own utility function in terms of energy dissipation costs and other related parameters [3]. The cloud data centre provider/manager allocates the cloud resources to cloud users dynamically. Machine learning (ML) techniques are very important for tackling scientific challenges in many fields such as agriculture, finance, automation, health, industry, etc. In fact, ML can be regarded as a sub-field of artificial intelligence (AI) [4]. An ML system, at first, makes observations of its surrounding environment and, then, improves its performance and efficiency for future tasks [5]. These observations are supported by data, and the sensors are the primary sources of the data. After analysing the mentioned input data, an ML algorithm generates and possibly updates a model from its surrounding/operational environment. From the communication system perspective, ML techniques analyse the traffic data and extract useful information/knowledge from them, which may not be available to humans by themselves [6]. Machine learning algorithms can be divided into four specific sub-types, which are: supervised learning [5], unsupervised learning [7; 8], semi-supervised learning [9], and reinforcement learning [10; 11]. In supervised learning algorithms, a function is learned that maps an input to an output based on example input-output pairs. After being trained on the training data, a supervised learning algorithm produces a function model. Then, the new examples can be mapped by this model [5]. If the data samples are not labelled or classified, unsupervised learning methods must be adopted for data pattern extraction. In another word, patterns that exist and are hidden within datasets can be identified by unsupervised learning algorithms [12; 13; 14]. In the machine learning literature, there exists an intermediate reasoning method, which is called reinforcement learning (RL).