## 5 Results and Analysis
### Case Studies and Results  
The proposed methodology has been implemented using the data set given in [31]. The electricity supply company of the Malaysian city of Johor generated these hourly load data. This data set shows the variation of load corresponding to day, month, year, time and temperature. The best correlation factor is obtained from combining all the input parameters to obtain the output. A combination of all input parameters is 93.8 percent correlated to the output. The methods have also been tested using the data of [32; 33] with similar input parameters. While investigating the data, it was found that around 2.5% of the data set was outliers. These outliers were removed. The output load data were scaled between 0 to 1 with the base quantity as the peak value. Hold-out validation was used with 80 and 20 for testing and training, respectively. The data set was trained with algorithms consisting of Ensemble, DTs, Neural Networks, SVM, GPR, and Linear Regressions. The output of DTs is shown in Figure 9. Figure 9 represents the true versus the predicted values of the system, and Figure 9 indicates how fit the model is and is used for the calculation of \(R^{2}\). If there is an overfitting issue, it has been crosschecked using cross-validation, replacing it with hold out validation. Similarly, the corresponding results of Linear Regression, SVM, Ensemble, GPR and Neural Networks are shown in Figures 10-14, respectively.  
Detailed statistical analyses of all the methods are shown in Table 2. It can be observed that the best method was found to be a Decision Tree. The output of the Decision Tree was further optimized with the objective to reduce MSE, and hyperparameters were further tuned. Three different methods--Bayesian Optimization, Grid Search, and Random Search--were simulated and compared. A detailed plot for hyperparameter tuning is presented in Figure 15. Parameters considered for the tuning of Decision Trees were leaf size and number of iterations. The number of iterations is significant with respect to the training time of the method. The optimal hyperparameters are shown in Table 3. It can be observed from Table 3 that the best method for hyperparameter tuning on this data set is Grid Search with a leaf size of 6. Results of all hyperparameter tuning of all the methods can be referred to in Table A1.  
Figure 9: Results with Decision Trees. (**a**) Actual versus predicted values of Decision Trees. (**b**) Linearity analysis of Decision Trees.  
Figure 10: Results with Linear Regression. (**a**) Actual versus predicted values of Linear Regression. (**b**) Linearity analysis of Linear Regression.  
Figure 11: Results with Support Vector Machines. (**a**) Actual versus predicted values of Support Vector Machines. (**b**) Linearity analysis of Support Vector Machines.  
Figure 12: Results with Ensemble. (**a**) Actual versus predicted values of Ensemble. (**b**) Linearity analysis of Ensemble.  
Figure 13: Results with Gaussian Process Regression. (**a**) Actual versus predicted values of Gaussian Process Regression. (**b**) Linearity analysis of Gaussian Process Regression.  
Figure 14: Results with Artificial Neural Network. (**a**) Actual versus predicted values of Artificial Neural Network. (**b**) Linearity analysis of Artificial Neural Network.  
Thus, three different methods to tune hyperparameters were analyzed to understand the functionality of tuning.