# Optimizing Energy Efficiency in Cloud Data Centers:
## 2 Background  
In the following sections, we will implement and evaluate several models from deep learning (DL) and reinforcement learning (RL). Before proceeding, it is useful to define these approaches within this framework to better understand their practical application to our problem. * **Random forests:** Random forests are a robust and versatile supervised learning method that combines several decision trees to improve the quality of predictions. During training, several decision trees are generated and their results aggregated, either by majority voting for classification tasks or by averaging predictions for regression tasks. By leveraging this ensemble approach, random forests improve accuracy and reduce the risk of overfitting. Thanks to their ability to handle large-scale, complex datasets, they are widely used in applications such as feature recognition, economic forecasting, medical diagnosis, predicting CPU utilization [11], and many other fields [12]. * **Self-organizing maps (SOMs):** The self-organizing map (SOM) is a type of artificial neural network designed to function in an unsupervised manner. It is effective for both dimensionality reduction and clustering tasks. SOMs project high-dimensional data onto a lower-dimensional (typically 2D) grid, revealing the underlying structure and relationships within the dataset. Competitive learning is used, in which grids deform at the input, where grid neurons compete to represent data. SOMs improve understanding of structures and relationships in data, with all their benefits for data visualization, pattern recognition, and other exploratory activities [13]. * **Firefly:** The Firefly optimization algorithm is a metaheuristic inspired by the flashing behavior of fireflies [14]. It uses the fireflies' brightness level to indicate candidate solutions in which brighter fireflies attract others, causing them to move toward better solutions in the search space. Two key parameters are the attractiveness coefficient and the light absorption coefficient; the two guide the fireflies towards the best possible solutions with minimal effort [15]. The total time complexity of the Firefly algorithm is as follows: \(O(t*n*n*d)\), where \(t\): The number of iterations the algorithm runs. \(n\): The number of fireflies in the population. \(d\): The number of dimensions in the search space. * **Q-learning:** Q-learning is a reinforcement learning algorithm that tries to select the optimum action according to the current state of the environment. It is a model-free algorithm. The basic concept of Q-learning is to learn a policy that tells an agent what action to take under what circumstances. In order to achieve this, a Q-function is formed that computes the utility or expected reward from performing a certain action in a certain condition and applying the optimal policy afterward. The Q-function is updated applying this formula [16]. \[Q(s_{t},a_{t})=(1-\beta)\times Q(s_{t},a_{t})+\beta\times(r_{t}+\lambda\times \max Q(s_{t+1},a_{t}))\] (1) where \(Q(s_{t},a_{t})\) represents the current Q-value for state \(s_{t}\) and action \(a_{t}\). \(\beta\) is the learning rate, indicating how quickly data on new states will be considered in subsequent steps \(\beta\in[0,1]\). \(r_{t}\) is the reward received after taking action \(a_{t}\) in state \(s_{t}\). \(\lambda\) is the Discount factor, a parameter \(\lambda\in[0,1]\) that determines the importance of future rewards. \(\max\)\(Q(s_{t+1},a_{t})\) is an estimation of the optimal Q-value in the future. \(\bullet\) If \(\beta=0\), the agent does not learn to improve its future actions. \(\bullet\) If \(\beta=1\), the agent relies entirely on the most recent experiences to guide its actions. \(\bullet\) If \(\lambda=0\), it means that the agent is only concerned with immediate rewards and does not consider future rewards at all. * If \(\lambda=1\), this means the agent fully considers future rewards, values them equally to immediate ones, and treats them with the same importance as immediate rewards.