# Benchmarking of Load Forecasting Methods Using Residential Smart Meter Data
## 2 Materials and Methods
### Forecasting Approaches
#### 2.2.3 Artificial Neural Networks  
This subsection details some fundamental concepts of ANN and its application for load forecasting. Moreover, how ANN-based models were used in this case study is presented. ANN, inspired by the neural processing commonly found in the human brain and its ability to learn, still have a significant number of practitioners in the function approximation field. Some of the advantages that can be pointed are the appropriateness of multivariate models and the ability to capture potential nonlinear relationships between output(s) and inputs during the learning process using some historical data (the training phase) [5; 8; 11; 18; 19; 26; 27]. At the training phase, the forecast error minimization is achieved through an iterative network adaptation. Therefore, the available and chosen historical data representativeness is crucial and the result is an automatic mapping of the relations that may exist between output and inputs. Another advantage that can be pointed to neural networks is the ability to deal with noisy data, whenever networks used are not too complex. The advantages of using neural networks for electrical load forecasting (possible applications to estimate active power demand, total electricity consumption or even power losses) are essentially the extraction of nonlinear relations that are present and the use of multivariate models. Since the training phase is based on a machine learning process, it is strongly dependent on the quality of historical data; otherwise the training phase may not contemplate the necessary, accurate and updated information to adapt the neural network. In load forecasting applications, the multilayer feed-forward architecture is still the most common neural model used. Each neuron, as the basic component of the architecture, processes the captured information in its inputs, such as the system's inputs or the previous hidden layers' outputs, by applying an activation function to the weighted sum of the inputs plus a scalar bias. Commonly, the selection of the activation functions depends on the type of forecasting [26; 27], with the most popular functions being the hyperbolic tangent function (_tanh_), the sigmoid function and the linear transfer function. The use of nonlinear functions is essential when there are nonlinear relations between outputs and inputs. At the network training phase, the network parameters (weights and biases) are adjusted to minimize deviations between measured and predicted values. Several training methods are commonly used based on iterative optimizing methods. The objective is to optimize the performance function, typically minimizing the mean squared error. More details about ANN architectures and mathematical foundations are better described in [26; 27].  
In this research, feed-forward neural networks were adopted, using one single hidden layer (more than one hidden layer were tested without increasing the overall performance in the test sample) [26]. To allow the comparison among different neural network dimensions, adapted to the context of each consumer's idiosyncrasies, it was decided to simulate a variable number of neurons (from 3 to 10, being the maximum adopted bearing in mind the input set dimension) [26]. For each scenario, 10 different trials were used in order to explore initialization of different random weights and biases. The best-case scenario (for a specific number of neurons and for a specific trial) was preserved and saved for each specific consumer. An example of an ANN model created is presented in Figure 3.  
ANN models in the scope of this research were created and tested using Neural Network package [28] available in the Scikit library of Python.  
Other approaches based on Support Vector Regression are also in the author's research domain. They were tested for this specific dataset, but created serious time constraints when tuning the hyperparameters for different consumers. Thus, this approach was not presented in this paper.