# Load Forecasting with Machine Learning and Deep Learning Methods
## 3 Methodology
### Modeling
#### 3.2.4 Multilayer Perceptron  
Feedforward artificial neural networks, mostly implemented using MLP, are composed of three types of interconnected layers: an input layer, where the input features are located; an output layer, where the forecasted variable is established; and some hidden layers, which correspond to the internal layers of the model. The connections between them are based on a weight structure, and the hidden layers are responsible for extracting patterns from the data and storing them during the training phase Each of the layers has its own neurons [60; 61]. The number of input neurons is normally equal to the number of features; the number of output neurons is the same as the number of variables to be predicted; and the number of hidden neurons, as well as the number of hidden layers, depend on the problem. As the hidden layers or hidden neurons increase, the ability of the model to extract hard patterns increases; however, the problem becomes more complex [62; 63]. This rule is not always fulfilled since as the number of hidden layers or hidden neurons increases, the model has a greater tendency to suffer from overfitting [64]. Therefore, the forecastedvariable depends on the variety of the hidden layers and hidden neurons. Therefore, similar to RF and the XGBoost, this is a black box model. In this case, the internal operations of the neurons depend on the activations of the preceding neurons. In (9), the process that governs the behavior of neurons can be seen in detail.  
\[z^{[i]}=w^{[i-1,i]}g\Big{(}z^{[i-1]}\Big{)}+b^{[i-1,i]} \tag{9}\]  
where \(z^{[i]}\) corresponds to the function of the studied neuron, \(g(\cdot)\) is the activation function of preceding neuron, and \(w^{[i-1,i]}\) and \(b^{[i-1,i]}\) are the weights of the neuron connection and bias, respectively.  
This process is propagated from the input layer to the output layer through the interconnections of the neurons. Furthermore, if a node has more than one input, the final value of the function of this node corresponds to the sum of the individual values of the functions of the node and their connections. The end of each step performed in the iterative training process is terminated by backward propagation, where the error is propagated to the input layers, and updating the values of the weights [65,66]. As in MLs, learning is based on minimizing a cost function \(J\). In this case, the cost function is governed by (10).  
\[J=\sum_{k=1}^{n}\frac{(g_{k}-y_{k})^{2}}{n} \tag{10}\]  
where \(y_{k}\) is the objective variable at time \(k\), and \(\hat{y}_{k}\) is the predicted series of the objective variable at time \(k\).  
MLP is a commonly used DL model. Its structure is beneficial for electricity consumption forecasting. The following are some important advantages of this technique:  
* Non-linear: It is capable of modeling non-linear relationships.
* Flexibility and adaptability: It has a high number of hyperparameters that offers this model flexibility in terms of network architecture and activation functions.
* Missing data: It can handle missing data effectively.
* Adaptability to changing patterns: It has the ability to learn and adapt to changes in complex patterns in data.