###### Abstract  
Intermittency in the grid creates operational issues for power system operators (PSO). One such intermittent parameter is load. Accurate prediction of the load is the key to proper planning of the power system. This paper uses regression analyses for short-term load forecasting (STLF). Assumed load data are first analyzed and outliers are identified and treated. The cleaned data are fed to regression methods involving Linear Regression, Decision Trees (DT), Support Vector Machine (SVM), Ensemble, Gaussian Process Regression (GPR), and Neural Networks. The best method is identified based on statistical analyses using parameters such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Square Error (MSE), \(R^{2}\), and Prediction Speed. The best method is further optimized with the objective of reducing MSE by tuning hyperparameters using Bayesian Optimization, Grid Search, and Random Search. The algorithms are implemented in Python and Matlab Platforms. It is observed that the best methods obtained for regression analysis and hyperparameter tuning for an assumed data set are Decision Trees and Grid Search, respectively. It is also observed that, due to hyperparameter tuning, the MSE is reduced by 12.98%. short term load forecasting; machine learning; Bayesian optimization; grid search; random search 2023202331612433316124331612  
The solution to STLF obtained using a Support Vector Machine (SVM) is shown in [9; 10]. Non Linearity aspects of the Input and Output of the data can be easily solved using SVM; however, sometimes SVM can be ineffective with large data sets [11]. The approach to STLF using Decision Trees (DT) is demonstrated in [12]. When the data set of the system is large, DTs can produce effective solutions. The disadvantage of using DTs includes false prediction through over-fitting [13]. Hidden relations between data points can be effectively calculated using Ensemble [14; 15]. Hardware integration of Ensemble methods is often costly due to its complexities [16; 17]. The issue of uncertainties can be dealt with using Gaussian Process Regression (GPR) effectively [18], but lacks a mechanism to find interrelation between parameters [19]. Neural Networks can provide accurate solutions to those problems, where data are changing continuously and can have good communication with Data Base Management Systems (DBMS) [20]. A comprehensive analysis of load forecasting methods using predictive models is given in [21]. Hyperparameter tuning plays a crucial role in optimizing the operating parameters of machine learning methodologies. This paper uses three different techniques for hyperparameter tuning using Grid Search, Random Search, and Bayesian Optimization. Grid Search optimization is usually used in medical applications. In this paper, it is utilized in STLF applications. The usage of Grid Search is broadly explained in [22]. The hyperparameter tuning for SVM is reported in [23]. The advantages of Grid Search methodology include its exhaustive nature of search but computational time increases exhaustively when a large number of hyperparameters are tuned [24]. Various studies observed Random Search as an important alternative to Grid Search. In Random Search, arbitrary combinations are used to find the optimal solution, hence to a certain extent it becomes independent of the number of hyperparameters [25]. One of the biggest limitations of Random Search algorithms is their high dependence on their randomness. Hence, different parameters produce different results [26]. The third method used for hyperparameter tuning in this paper is Bayesian Optimization. Bayesian Optimization is based on Bayes theorem, which is based on conditional probability. The goal of Bayesian reasoning is to become \"less