# Short-Term Power Forecasting Framework for Microgrids Using Combined Baseline and Regression Models
## 3 Methodology
### Neural Network Regressor  
Among the numerous types of existing ANNs [59], one of the most commonly used approaches for regression purposes is the MLPR [60]. It is a powerful tool for predicting continuous variables, thereby supporting multioutput regression. This architecture is selected for the proposed architecture due to its relative simplicity and because it is less computationally expensive than other ANN types, such as recurrent neural networks (RNNs).  
The basic structure of an MLPR with a single hidden layer can be seen in Figure 2. A similar structure is adopted if more than one hidden layer is added.  
For the proposed architecture, a multilayer perceptron (MLP) that conducts training using backpropagation [61] is used. The activation function in the hidden layers is the rectified linear unit (ReLU) function (1), which has a lower computational cost than other activation functions, such as the logistic or hyperbolic tangent functions. In contrast to the MLP used for classification, the activation function in the output layer is the identity function, achieving continuous values for the outputs (a regression behavior). Furthermore, the squared error is used as the loss function during training.  
\[f(x)=max(0,x), \tag{1}\]  
Thus, as is usually recommended, parameter regularization techniques are applied to obtain better generalized models [62] (reducing the probability of overfitting during the training process). This means that weights with large magnitudes are penalized during training, so large values are not retained (they are regularized). Two well-known techniques for this purpose are lasso regression (\(\ell 1\)) and ridge regression (\(\ell 2\)) [62]; the penalty term is squared in \(\ell 2\) (which brings higher sensitivity to high weight values). For this reason, the \(\ell 2\) technique is preferred. The regularization effect is adjusted by a hyperparameter called \"alpha\".  
Figure 2: Multilayer perceptron regressor with a single layer and multiple outputs.  
The use of an MLPR for modeling requires deciding the number of hidden layer neurons (hyperparameter \(n_{h}\)) to be used. In this sense, some authors have proposed heuristics (rules of thumb) to select the number of hidden neurons [34]. Their expressions are shown in (2)-(7):  
\[n_{h}=(2\cdot n_{i})+1 \tag{2}\]  
\[n_{h}=2\cdot(2\cdot n_{i})+1 \tag{3}\]  
\[n_{h}=\sqrt{n_{o}+n_{i}}+l \tag{4}\]  
\[n_{h}=\sqrt{\frac{N}{n_{i}\cdot log(N)}} \tag{5}\]  
\[n_{h}=\frac{n_{i}+n_{o}}{2}+\sqrt{N} \tag{6}\]  
\[n_{h}=\frac{n_{i}+n_{o}}{2} \tag{7}\]  
\(n_{h}\) indicates the number of hidden layer neurons; \(n_{i}\) is the number of input neurons; \(n_{o}\) is the number of output neurons; \(l\) is an integer between 1 and 10; and \(N\) is the number of training samples.  
Considering the characteristics of each input and output dataset (their sizes), it is possible to directly calculate the recommended number of hidden neurons according to these rules. These recommendations are used to obtain initial estimations of the range of \(n_{h}\) values that will be tested during the experiments.