## 3 Finding Models to Predict Energy Consumption
### Machine-Learning Models  
In addition to deep-learning models, we also propose two machine-learning models: SVR and Random Forest.  
Support Vector Machines (SVM) has been proposed as an alternative to traditional ANNs for classification and regression tasks. In particular, SVM provides better support for forecasting time series from nonlinear systems [37]. SVM is a machine-learning technique based on statistical learning theory [38]. Extant literature suggests that SVM performs well in forecasting time series [37; 39; 40]. Support Vector Regression (SVR) is a regression technique based on SVM [41]. The main differences relate to the formats and types of input and output. Kernel functions are used to map the data through nonlinear functions in an n-dimensional space. In this way, it is possible to transform nonlinear problems into linear problems. Research suggests that SVR presents accurate results for predicting energy consumption and, as such, is commonly used in the field [42]. Despite its advantages, the lack of predetermined heuristics for both the design and parameterisation of SVR models is a major drawback in using SVR [37]. As such, studies tend to be application-specific and to lack generalisability [37].  
Random Forest is a machine-learning technique based on different decision trees. Random Forests' implementation involves random selection of features based on the position of the root node. The model output consists of the average of the results for all trees. When compared to a single decision tree, Random Forest presents a better performance [43; 44; 45]. The greater the number of trees, normally the better the performance of the model but the slower the model and the more inefficient the real-time predictions. It is one of the most popular machine-learning techniques used for classification and regression problems [21; 46; 47]. Random Forest's popularity is often attributed to its higher accuracy when compared with ANNs and SVR [48].  
Machine-Learning Model Settings  
Similar to the deep-learning models, we also perform grid search to find the best hyperparameters of the machine-learning models, using the same data set splitting procedure. The hyperparameters used vary according to the technique (see Table 2). For SVR, cost and the type of kernel were used, whereas the maximum depth of trees and the number of trees were used for Random Forest.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Technique** & **Parameter** & **Levels** \\ \hline SVR & Number of C & 0.1, 1 and 10 \\ SVR & Type of kernel & Linear, polinomial and RBF \\ Random Forest & Number of max. depth & From 3 to 6, step 1 \\ Random Forest & Number of trees & From 50 to 200, step 50 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Machine-learning parameters and levels.  
Figure 5: Mean absolute error (MAE) grid search result for **(a)** simple RNN, **(b)** LSTM, and **(c)** GRU.  
Figures 6-8 present the grid search results for RMSE, MAPE, and MAE, respectively. For SVR, the best configuration across the three metrics used--RMSE, MAPE, and MAE--is represented by SVR-0.1-linear, for which the C value is 0.1 and used the linear kernel. For Random Forest, the models with configurations with (a) a maximum depth of three with 50 trees (Random Forest-3-50), (b) maximum depth of six with 50 trees (Random Forest-6-50), and (c) a maximum depth of six with 100 trees (Random Forest-6-100) generated the best results for RMSE, MAPE, and MAE, respectively. These four model configurations will be used in our benchmark evaluation.