# Forecasting Irregular Seasonal Power Consumption.
## 2 Materials and Methods
### Forecasting Methods
#### 2.2.1 Artificial Neural Networks  
Neural networks are computational models structured in the form of layers with nodes interconnected as a network. They are named because of their resemblance to the human brain structure. The nodes, called neurons, perform simple operations in parallel and are located in each of the layers of the network. The layers are of three different types: the input layer, where neurons receive direct information from the inputs; hidden layer (s), whose neurons use the information from the neurons of previous layers and feed the next layers; and the output layer, where neurons use the information from the hidden layers to produce an output. Thus, there is an input layer, one or more hidden layers, and an output layer. The connections between the different layers are made through the connection of their neurons, which are called synapses. The strength of the connection between neurons is determined by a weighting established at the synapse.  
The most suitable structure for forecasting time series is the NARX type structure [44; 45]. It is a recurrent dynamic neural network, with feedback connections. Figure 5 shows a close-loop representation of the NARX structure [46]. Neurons receive information from exogenous input variables in addition to the target series itself and the feedbacks. In order to improve forecasts, it can be used the past predicted and observed values delayed through a tapped delay line (TDL) memory. The circles after the input layers denote the TPL delay (e.g., one to two delays in the figure).  
The input variables \(x_{t}\) are exogenous variables used in the model. Both \(x_{i}\) and \(y_{t}\) are connected by axioms to which weights \(w_{i}\) are assigned, and with an activation function \(f\) that is integrated with an aggregation function \(\Sigma\). The output \(\hat{y}_{t+1}\) provides future forecasts after the network has been trained. \(b\) stands for the bias whose presence increases or decreases the neuron's processing capacity.  
Figure 5: NARX neural network schema. There is an input layer with variables, one hidden layer and one output layers. Circles represent tapped delay line (TDL).  
The mathematical and non-linear representation that governs the network is shown in (1), where \(x_{t}\) represents the inputs and \(y_{t}\) the objective function, while \(\hat{y}_{t+1}\) represents the prediction. \(D_{x}\) and \(D_{y}\) are the time delays applied in the network.  
\[\hat{y}_{t+1}\ =f\Big{[}x_{t},x_{t-1},\ldots,x_{t-D_{x}+1},y_{t},y_{t-1}, \ldots,y_{t-D_{y}+1}\Big{]}. \tag{1}\]  
The NARX neural network maps the function through the multilayer perceptron, using the time delays for both the input variables and the output feedback [47].  
An alternative to this neural network is a function fitting neural network. This is a type of shallow neural network based on multilayer perceptron (MLP) with which we can make adjustments to non-linear functions (non-linear regression, NLR). The use and application of such a network for the prediction of electricity demand has been discussed previously [48]. The mathematical representation that governs this network is shown in (2).  
\[\hat{y}_{t+1}\ =f[x_{t},x_{t-1},\ldots,x_{t-D_{x}+1}]. \tag{2}\]  
Here \(x_{t}\) are the predictors, which are several variables (including the observed values of the time series) used to feed the model. A representative schema for this neural network is shown in Figure 6.  
By training the network, weights are assigned to the synaptic connections, minimizing an error criterion. The ANNs used in this work are trained using the Levenberg-Marquardt algorithm [49], and minimizing the mean squared error (MSE). After the training process, to give the predictions, a closed loop network is performed, and forecasts are provided.