## 2 Methodology
### Ensemble Model Creation  
Given the complexity and variability of the electrical load data, ensemble models were created to enhance predictive accuracy. Ensemble methods combine the predictions of multiple models to reduce overfitting and improve the overall resilience and accuracy of the resulting model. The AutoML frameworks were utilized to automate the creation and tuning of these ensemble models, based on the settings of each framework. Several methods for making ensemble models exist, with different approaches as far as the final resulting model is concerned, as mentioned in [27]. Bagging is a method that involves training multiple instances of the same model on different subsets of the given dataset. These subsets are created through bootstrapping, a process where random samples are drawn with replacements from the original data. The purpose of bagging is to reduce variance and improve the accuracy of predictions. In the case of regression, the final prediction is the average of the individual model predictions, while for classification tasks, it is determined by a majority vote among the models. Boosting is an ensemble technique that sequentially trains models, with each new model in the sequence focusing on correcting the residual errors made by the previous models. Unlike bagging, where models are trained independently, boosting models work together in a dependent manner, where each model improves upon the mistakes of its predecessor. This approach helps in reducing bias and improving the overall accuracy of the model, always with respect to the previously trained model. Stacking is an ensemble method that combines predictions from multiple models, known as level 0 models, using a meta-model, or level 1 model. In this technique, the meta-model is trained on the predictions generated by the base models rather than on the original training data. The meta-model then learns how to best integrate the outputs of these individual models to generate a final prediction. Stacking leverages the strengths of different models to improve predictive performance, often yielding better results than using a single model alone. Automated Feature Engineering involves the automatic generation, selection, and transformation of features from raw data to enhance model performance. This method leverages domain knowledge and data patterns to create new features that can be more informative for the predictive model. By automating this process, AutoML systems can identify complex relationships within the data and produce features that significantly improve the accuracy and robustness of machine learning models [28]. Hyperparameter optimization is a crucial component of AutoML, focusing on the automatic search for the best set of hyperparameters that enhance model performance. The process involves exploring various configurations of hyperparameters, such as learning rates, regularization parameters, and tree depths, to identify the optimal combination for a given model and dataset. By automating hyperparameter tuning, the frameworks can achieve significant improvements in model accuracy and efficiency without requiring extensive manual intervention [29]. Mathematical Formulation  
As studied in [30], one form of AutoML includes the ensemble method creation by aggregating predictions from several base models to form a final prediction. This generic approach is applicable to various ensemble techniques, particularly those that use weighted averaging of model outputs. Let \(f(x1)\), \(f(x2),...\), \(f(xn)\) represent the set of predictions from \(n\) different base models for a given input \(x\). The combined prediction \(F(x)\) of the ensemble model can be represented as follows:  
\[(x)=\sum_{1}^{n}w_{i}\times f_{i}(x) \tag{1}\]  
where \(f_{i}(x)\) is the prediction from the \(i-th\) base model for input \(x\); \(w_{i}\) is the weight assigned to the \(i-th\) base prediction of the model. The weights \(w_{i}\) are non-negative values with summation: \(\sum_{i=1}^{n}w_{i}=1\).