Power consumption model based on feature selection and deep learning in cloud computing scenarios

IET CommunicationsVolume 14, Issue 10 p. 1610-1618 Research ArticleFree Access Power consumption model based on feature selection and deep learning in cloud computing scenarios Yang Liang, Yang Liang School of Computer Science and Engineering, Central South University, Changsha, People's Republic of China School of Informatics, Hunan University of Chinese Medicine, Changsha, People's Republic of ChinaSearch for more papers by this authorZhigang Hu, Corresponding Author Zhigang Hu zghu@csu.edu.cn School of Computer Science and Engineering, Central South University, Changsha, People's Republic of ChinaSearch for more papers by this authorKeqin Li, Keqin Li Department of Computer Science, State University of New York, New Paltz, NY, USASearch for more papers by this author Yang Liang, Yang Liang School of Computer Science and Engineering, Central South University, Changsha, People's Republic of China School of Informatics, Hunan University of Chinese Medicine, Changsha, People's Republic of ChinaSearch for more papers by this authorZhigang Hu, Corresponding Author Zhigang Hu zghu@csu.edu.cn School of Computer Science and Engineering, Central South University, Changsha, People's Republic of ChinaSearch for more papers by this authorKeqin Li, Keqin Li Department of Computer Science, State University of New York, New Paltz, NY, USASearch for more papers by this author First published: 01 June 2020 https://doi.org/10.1049/iet-com.2019.0717Citations: 4AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract High power consumption of cloud data centres is a crucial challenge in modern cloud computing. To comply with the conceptions of green computing, power consumption prediction of the computing cluster has a major role to play in these energy conservation efforts. However, due to complexity and heterogeneity in cloud computing scenarios, it is difficult to accurately predict the power consumption using conventional approaches. To this end, this study presents a power consumption model based on feature selection and deep learning to powerfully cope with low energy efficiency. Different from other methods focusing on only a few performance attributes, the proposed method takes into account up to 12 energy-related features and introduces deep neural network architecture, aiming at making full use of massive data to train model completely. In particular, this approach is composed of three main phases including (i) performance monitoring and energy-related feature acquisition, (ii) essential feature selection, and (iii) model establishment and optimisation. Representative results of comprehensive experiments, in terms of the relative error, reveal that the proposed power consumption model can undoubtedly achieve state-of-the-art predictive capability when compared with other models in most cases. 1 Introduction Over recent years, the scale of the data centre (DC) is increasingly expanding, with the accelerated growth of data generated by applications in various industries. Cloud computing is playing an increasingly important role in our daily lives, driving the energy consumption (EC) of DC to new highs. The energy management issue of DC has drawn people's attention extensively [1, 2]. The industry generally uses power usage effectiveness (PUE) to measure the energy efficiency of DC. As an effective engineering ratio, PUE is shown in the following equation: (1)where the numerator denotes the power consumption of all facilities in the DC including mechanical devices, electrical equipments, IT infrastructures, and other necessary devices, while the denominator denotes the power consumption of IT infrastructures, providing real useful IT work. The closer the value of PUE is to 1, the higher the energy efficiency in DC is. According to several DC industry surveys, provided by Uptime Institute as well as other professional consulting institutions, the average power usage effectiveness (APUE) has only improved from 2.5 in 2007 to 1.89 in 2011 while it only improved from 1.89 to 1.7 in 2014 [3]. In the USA, the APUE is still around 1.7 in 2017 for enterprise DCs, while it is between 1.5 and 1.6 for the newly built DCs. Very few advanced DCs, such as those of Facebook and Google, can achieve PUE of 1.2 or lower. However, it is obvious that the improvements of energy efficiency measured by PUE are slowing down significantly. Even as excellent as Google, it is getting harder and harder to further reduce the PUE value. As after a certain degree, the interaction between refrigeration and electrical systems and complex feedback loops make it difficult to accurately use conventional engineering formulas to derive and optimise DC efficiency [4, 5]. Thus, developing innovative methods or strategies for reducing PUE, that is minimising EC and operational cost eventually, deserves our endeavours. Over the last decade, energy efficiency is a major research concern in modern cloud computing [6]. Many works, some of which will be discussed in Section 2, provide various solutions to estimate power consumption of a dynamic and complex DC. Some researchers have employed machine learning (ML) techniques, driven by data essentially, to resource provision and management in cloud DC for ensuring energy efficiency [7-10]. ML is based on training data to build statistical models, helping us to make decisions or predictions about events in the real world. There are many traditional models including support vector machines, decision tree, KNN, naive Bayes, K-means, neural networks (NNs), and so on, which belong to ML. In this paper, we present an improved deep learning (DL) framework, driven by massive data, to offer a solution for improving energy efficiency by predicting next energy demand in the cloud. Considering the non-linearity and complexity of various devices in DC and accuracy of the experiment [11], we focus on researching the power consumption of IT infrastructures, which is expressed as in (1). To this end, our goal is to improve the energy efficiency of cloud computing clusters by allocating resources legitimately using our power consumption model. Although modelling and predicting real-world dynamic system behaviours have received widespread research interest, approximating any non-linear or complex system remains a challenging task. To address these challenges, the significant contributions of this work can be summarised as follows: i) A deep NN architecture is elaborated and a power consumption model based on feature selection and DL framework is proposed, achieving a boost in prediction accuracy regardless of the application belonging to I/O intensive, compute-intensive or transactional web. ii) An elaborate-designed data preprocessing process including performance monitoring, energy-related feature acquisition, and key feature selection has been provided, taking into account more comprehensive energy-related features and removing redundant features effectively based on information theory. iii) Extensive and multi-angle experimental analysis are implemented for training and validating the mathematical model, as well as the research domain accredited benchmarks are used for data collection and feature extraction by performing tasks of different application scenarios. iv) Analyses and comparisons between six different power consumption models in terms of the relative error (RE) demonstrate the proposed model is effective and vastly superior. The rest of the paper is organised as follows. Cloud computing energy efficiency related works are reviewed in Section 2. In Section 3 and 4, the well-designed modelling processes including data preprocessing and model constructing are described in detail. The analysis and comparison of experimental results is presented in Section 6. Finally, Section 7 summarises the paper with recommendations for future work in this area. 2 Related work No doubt that energy efficiency in cloud computing has become a hot theme in recent years [12]. Many solutions have been carried out for modelling the aggregate power consumption of a server. As servers conduct most of the work in DCs, they are the most power proportional components in a DC. Roy et al. [13] present a server power model as a summation of CPU and memory power consumption, which is one of the simplest power models. Their power model was represented as (2)where and are EC of the CPU and the memory while running Algorithm 1. Considering more components of a server, Tudor et al. [14] propose a power model expressed as a function of energy used by CPU, memory, and I/O devices. Furthermore, Song et al. [15] describe a similar power model by expressing the system power consumption as a summation of CPU, memory, disk, and network interface card, which can be shown as (3)Another energy model can be further constructed considering the levels of resource utilisation by the key components of a server as [16] (4)where is the CPU utilisation, is the memory access rate, is the hard disk I/O request rate, and is the network I/O request rate. refers to the predicted power consumption of server at time t while , , , and are the coefficients of CPU, memory, disk, and NIC, respectively. Tian et al. describe the power consumption of a server as follows [17]: (5)where denotes the service rate and represents the utilisation of server i. is the static power consumption of server i. , , are constants determined by the DC. A different version of model is created by Mills et al. to estimate the system EC [18]. The power consumed by a server in this power model is given by (6)where stands for overhead power which is a fixed factor of the power consumed when the CPU is operating at full speed. denotes the execution speed of the processor. The overhead includes the power consumption by all other system components such as memory, network, and so on. What is more, in recent years, numerous studies of improving energy efficiency in DCs have been performed [19-22]. In terms of the results, most researchers achieve up to a large proportion of reduction in energy usage. However, the studies for predicting power consumption is insufficient while power consumption estimation is critical to resource provisioning and task scheduling [23-25]. According to different application scenarios, Zhou et al. [26] describe the mathematical expressions of the power regression model in detail and verify that the power regression model has higher precision than other regression models (linear regression, exponential regression, and polynomial regression). Zhang et al. [27] find that the cubic polynomial model could obtain better results whether EC swing in a small range or a wide range, exceeding the linear model that is only suitable for a small range. Bertran et al. [28] prove that the performance monitoring counters (PMC) power model still has good performance, whose average error is less than 3%. Park and Mun [29] propose a prediction approach that power consumption is determined by basic power and active power, not depending on actual power consumption. The basic power is constant determined by hardware while the active power can be easily calculated with the help of Cloud Monitor. The results show that average error rate is about 4.22% in the CPU test. Liu et al. [30] present an abstract energy consumption (AEC) model to estimate EC of the cloud computing. Particularly, considering the source-level EC of a task is not the simple summation of that of its statements, two quantitative measurements, 'cross-degree' and 'reuse-degree', are proposed as the code structure features. Experimental results show that the mean deviation between the EC and AEC is around 0.005. Yu et al. [31] construct a CMP EC model by only choosing some dominant parameters such as CPU, memory, disk, and so on. Moreover, taking into account different scenarios, they mainly focus on EC changes of computer systems in data-intensive and compute-intensive states. Experiments show that the CMP model can achieve a higher accuracy on power estimation than FAN and Cubic mode, especially in data-intensive scenario. Foo et al. [32-34] develop an EC prediction model based on an evolutionary NN combining with several novel mechanisms of a genetic algorithm. The results, both in terms of forecasting speed and accuracy, suggest that the evolutionary NN approach to EC forecasting for cloud computing is highly promising. As ensuring energy efficiency in DCs is a worthwhile endeavour, more and more innovative approaches, such as machine learning, are applied to resource conservation and management problems in the cloud computing [9]. Given the complexity of distributed clusters and the interaction among multiple control systems [35], accurate energy efficiency models allow DC managers to optimise operational configurations without on-site commissioning. Inspired by the challenges and deficiency of the existing works, we present a power consumption model based on feature selection and deep learning (abbreviated as FSDL model) in cloud computing scenarios. While drawing on this idea from aforementioned efforts, our goal is quite different because we seek not only energy efficiency but also the high prediction accuracy of power consumption, leading to novel methodology. Section 3 and Section 4 will elaborate the procedures of our approach. Section 5 reveals the comprehensive experimental results. 3 Methodology The methodology is mainly composed of the following stages: performance monitoring, energy-related feature acquisition, feature selection, building a power consumption model, and evaluation of experimental results. Fig. 1 shows each step of constructing a power consumption model by the methodology proposed in this paper. Fig. 1Open in figure viewerPowerPoint Overall framework of model construction The performance monitoring step is responsible for monitoring all relevant performance metrics as the primary data source. Then energy-related features are obtained and formatted. At the stage of essential feature selection, we calculate the importance ratio of each feature based on information entropy theory so that core features can be selected. Based on above, we build a power consumption model based on DL approach and optimise the parameters. Finally, we evaluate the results of the experiment and demonstrate the effectiveness of the proposed model. 4 Data preprocessing As one type of ML approaches, DL is evolved from the deep research in NNs while its network structure is more complex than other ML models. DL is used to learn the essential characteristics and laws of large amount of sample data entered. Therefore, the quality and quantity of sample data play a crucial role in training the DL models. In this work, we first designed the process of data preprocessing, specifically including energy-related feature acquisition and essential feature selection, to ensure the quality of the training data set. 4.1 Performance monitoring and energy-related feature acquisition Data acquisition is an important part of performance monitoring and analysing in cloud computing, and it is the foundation of data processing, analysis, and display. Moreover, as the MapReduce parallel computing framework is a widely used programming model in DCs, it has a significant impact on the EC of the DC. Hence, experimental data sets are collected from the Hadoop cluster while the MapReduce jobs are running. To obtain the working status of the servers and collect experimental data in time, it is extremely significant to monitor and manage the performance metrics of the server cluster. To the best of our knowledge, many commonly used performance monitors, i.e. Ganglia, Hadoop build-in counters, Nagios, Nigel's performance Monitor (nmon), Cloudera Manager, Zenoss, Zabbix, and so on, can contribute to data monitoring and acquisition. For instance, Ganglia, a scalable distributed monitoring system for high-performance computing systems, is mainly used to collect system-level information including CPU utilisation, disk utilisation, and so on. Nagios, an open-source application, can monitor systems, networks, and infrastructure. Besides offering host monitoring, Nagios can provide an anomaly detection mechanism when things go wrong. Zabbix, an open-source monitoring tool for networks, operating systems, and applications, can monitor statistics such as CPU load, network utilisation, disk space, and so on. However, most of them can only monitor some basic performance metrics and cause excessive system overhead. Considering the limitations and defects of a separate performance monitoring tool in cloud computing platforms, our inspiration was to design a comprehensive solution for energy-related features acquisition, combining several typical and complementary open-source solutions. To achieve a trade-off between the system overhead and the scope of monitoring metrics, we proposed a combined monitoring method consisting of Ganglia, Hadoop build-in counters, and Zabbix in this work. The main advantages of these components are as follows: Ganglia can monitor basic performance metrics of a cluster in real time, with low system overhead and no impact on the performance of related services; Hadoop build-in counters are mainly used to report multiple indicators related to MapReduce jobs and the results do not need to be transmitted over the network; Zabbix supports secondary custom development to monitor various required performance parameters. This combined monitoring approach effectively fused the strengths of these three components, not only co-monitoring more diverse energy-related metrics but also ensuring a lower overall system overhead. The required energy-related metrics and the corresponding monitoring ways are shown in Table 1. Table 1. Energy-related metrics and corresponding monitoring ways Number Metrics Unit Description Acquisition method 1 CPU utilisation percent percent of time that the CPU is utilised Ganglia 2 average load loads/procs average number of processes simultaneously in ready state during the last minute Ganglia 3 memory use gigabytes memory use for processes Ganglia 4 map task read gigabytes data read by map task from local disk Hadoop built-in counters 5 reduce task read gigabyte data read by reduce task from local disk Hadoop built-in counters 6 map task write gigabyte data written by map task to local disk Hadoop built-in counters 7 reduce task write gigabyte data written by reduce task to local disk Hadoop built-in counters 8 shuffle size gigabyte data transferred from map to reduce Hadoop built-in counters 9 network I/O speed gigabytes/s data transmitted and received Ganglia 10 file size gigabytes size of MapReduce jobs Hadoop built-in counters 11 number of M/R instructions number instruction number of jobs Zabbix 12 average jobs completion duration hour time taken to finish a MapReduce job Hadoop built-in counters 13 disk utilisation percent percent of time that the disk is utilised Ganglia 14 transmission and read/write ratio percent data transfer and read/write ratio Zabbix 15 available space in file system gigabytes file system free space Zabbix 16 page faults/s number speed of page errors caused by executing threads in the process Zabbix 17 bytes consumed per CPU second bytes average number of bytes consumed per CPU second Zabbix 18 context switches rate number number of switches between processes or threads per second Zabbix 19 HDFS R/W throughput bytes/s amount of data successfully transmitted per second in HDFS Hadoop built-in counters 20 disk traffic bytes/s rate of disk transfers Zabbix 21 paging rate number number of pages READ to physical memory or written to pagefile(s) per second Zabbix 22 power consumption kw power consumed by Hadoop cluster smart meter 4.2 Essential feature selection Theoretically speaking, all energy-related features need to be monitored and recorded. Nevertheless, there are huge differences between different attributes for the importance of decision making, even affecting the correctness of decision. During data acquisition and performance monitoring, data sets usually have problems such as irrelevant attributes and redundant attributes [36]. To solve the problem of effective information redundancy in high-dimensional data, we will discuss some classical data dimensionality reduction algorithms and the process used to select a subset of relevant features from those extracted in the preceding section. As one of the most widely used unsupervised dimensionality reduction algorithms, principal component analysis (PCA) [37] obtains the projection matrix by maximising the variance among samples after dimensionality reduction, so as to preserve the distribution properties of global samples as much as possible. However, the PCA algorithm consumes a large amount of memory and has a high time complexity. As a result, it is difficult to calculate the eigenvectors of high-dimensional data, and the processing time cannot meet the requirements of related applications. Linear discriminant analysis (LDA) [38], a supervised linear dimensionality reduction algorithm, can obtain a projection subspace with strong discriminative ability by maximising the quotient between inter-class dispersion and intra-class dispersion to separate the samples of different classes as far as possible. However, LDA focuses on the larger inter-class dispersion and ignores the smaller inter-class dispersion, which is likely to cause the fusion of different classes that are close to each other in the subspace, resulting in excessive reduction and information loss of the subset. Different from PCA and LDA, locally linear embedding (LLE) [39] belongs to the category of non-linear dimensionality reduction methods. Under the premise of keeping the original data properties unchanged, LLE can leverage local linearity to represent global non-linearity and map data from high-dimensional space to low-dimensional space to determine the subset features. However, LLE is sensitive to the selection of the nearest-neighbour number, as different nearest-neighbour numbers have a great influence on the dimensionality reduction results in the high-dimensional space, which will lead to a large fluctuation in the feature selection results. Therefore, considering the problems existing in the traditional data dimensionality reduction algorithm above, we have to develop a more effective, targeted attribute reduction mechanism to optimise the feature set of Table 1. Concretely, we will calculate the importance ratio of each feature, based on information entropy of rough set theory, to remove redundant attributes to streamline the dimension of the feature vector. Definition 1.(Feature selection): Select M(M<N) sub-features from N features. Among the M sub-features, the objective function can achieve the optimal solution.Selecting as few sub-features as possible will not significantly reduce the effect of the model. Conversely, building a model by selecting partial core sub-features can greatly reduce the execution time of the learning algorithm and improve the efficiency of the model.Information entropy is the average rate at which information is produced by a stochastic source of data [40]. Information entropy is a measure of the degree of confusion or dispersion of distribution. The more dispersed the distribution (or the more even the distribution) is, the larger the entropy gets. By contrast, the more orderly the distribution (or the more concentrated the distribution) is, the smaller the entropy becomes. Definition 2.(Information entropy): Let category be X and the corresponding label be x, x ε X. Let P(x) denotes the probability of x; then the information entropy of X is defined as (7)where b is the base of the logarithm used. In (7), b can be set as 2 as the entropy is typically measured in bits. The measure of information entropy associated with each possible data value is the negative logarithm of the probability mass function for the value.In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable Y given that the value of another random variable X is known. The entropy of Y conditioned on X is written as . Definition 3.(Conditional entropy): Let x be the value of attribute X and let y be the value of attribute Y. If is the entropy of the discrete random variable Y conditioned on the discrete random variable X taking a certain value x, then is the result of averaging over all possible values x that X may take. Then the conditional entropy can explicitly be written as (8) Definition 4.(Kullback–Leibler divergence): the Kullback–Leibler divergence (KLIC) is a measure of how one probability distribution diverges from a second, expected probability distribution. Then the KLIC of attribute A is defined as the difference between the basic entropy and the conditional entropy of its attribute (9)where is the basic entropy, while is the conditional entropy under the condition of attribute A. The greater the attribute's KLIC is, the more information the attribute can provide. Thus, KLIC describes the purity of an attribute.According to (7) and (8), assume that C is the collection of load performance attributes collected from the operating system, and D is the cluster power consumption value when each feature value is acquired during the execution of jobs. Then the conditional entropy of each performance attribute can be expressed as (10)In cloud computing, various performance attributes have different effects on power consumption of computing clusters. Our goal is to apply information theory to detect the 'weightier' input features that contribute positively to the power consumption. From this perspective, assume that is an arbitrary attribute in the conditional attribute set C, then its importance relative to the system power consumption can be described as the entropy change of C relative to decision attributes before and after is removed from C. Then the importance of is quantified as (11)Here, we can achieve the purpose of attribute reduction by calculating the importance ratio of each attribute in C according to the above equations. 5 Power consumption model implementation DL approach can effectively model and predict non-linear dynamic systems such as the cloud DCs, which is a viable solution for reducing power consumption in cloud computing. 5.1 Deep neural network architecture (DNNA) DL allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction [41]. As a multilayered complex NN structure, the DL model can effectively solve the problem that the shallow network usually cannot converge to the global optimal solution. The DNNA, a framework used for power consumption modelling, consists of three main components: the input layer, the hidden layer, and the output layer. The input layer contains d input nodes, depending on the dimension of the input feature vector, and 1 bias node, whose common value is 1. The hidden layer usually has m layers and each layer has k hidden nodes and 1 bias node, while the most appropriate values of m or k depend on the comparison of multiple experiments. The output layer has only 1 output node representing the power consumption. Fig. 1 illustrates the structure of DNNA. In the DNNA we specify a set of n vectors FV1, FV2, …, FVn, where each vector FVi∈Rd (d-dimensional of feature vector space). EPC, an abbreviation of electrical power consumption, denotes the result estimated by the entire DNNA at the output layer. In a forecasting process using DNNA, specifically, the input matrix X, consisting of w feature vectors, is multiplied by the model parameters matrix θ to produce the first hidden state matrix H1. The next hidden layers continue to forward propagate intermediate results using the same computational process until the predicted power consumption value is calculated. 5.2 Mathematical model Fig. 2 presents the elegant design of the DL architecture which can search for relationships between complex input features and power consumption by training massive amounts of data. Understanding the underlying mathematical behaviour of DNNA allows us to control and optimise it. Fig. 2Open in figure viewerPowerPoint Deep neural network architecture Let the DNNA input layer comprise of d input eigenvalues denoted as . Starting from the second layer, specially, is the input of each neuron that can be generated by (12)where is the connection weight between the input layer and the first layer of the hidden layers. i identifies the neuron of the preceding layer while j identifies the neuron of the current layer. is the weighted sum of the outputs of all neurons in the preceding layer. Then the output of the jth neuron is determined by an activation function shown in (14) acting on it (13)Equation (13) is the sigmoid function usually used as an activation function. The argument z will receive an assignment from , the input variable of the jth downstream neuron. Assume there are m l