# Prediction of Building's Thermal Performance Using LSTM and MLP Neural Networks
## 2 Materials and Methods
### Artificial Neural Networks Developed
#### 2.1.1 Multi-Layered Perceptron Neural Networks  
In this work the structure of the MLP neural network is variable due to the use of different numbers of time lags. Depending on the structure of these artificial neural networks, different network models can be generated. The most commonly used is the so-called feed-forward model. This model is composed of several layers (see Figure 1). The first layer, or input layer, is where the model inputs are introduced, whereas the last layer, or output layer, is where the results of the trained network are given. In addition, among them, the number of intermediate layers (hidden layers) can be zero, one or more [18, 20]. The particular network architectures are differentiated by the number of hidden layers and hidden neurons according to the complexity of the problem [45, 46]. The grid of tested values for the hidden layers was between 0 and 4. In contrast, the grid of values for the hidden neurons was composed of specific numbers based on the formulas presented by [PERSON] [47], [PERSON] et al. [48] or [PERSON] et al. [49], which take into account the size of the sample and the number of inputs and outputs of the network. Furthermore, in this case, due to the complexity of the problem, this grid of neurons had to be expanded with the values: 50, 100, 200, 500 and 1000.  
The MLP neural network training is done with a backward propagation algorithm; errors are propagated through the network and adapted to the hidden layers. Thus, error-correction learning (actual system responses must be known) is used to train the ANN [29, 50]. There are several ways to perform the training [19, 51] but, the methodology used in this work consists of updating the weights with an average update of the weights (batch learning), which is achieved by incorporating all the patterns in the input file (an epoch) and accumulating all the weight updates. There is also a need for a stop criterion [52, 53]. Although there are other options, such as a threshold for the mean square error (MSE) or a limitation on the maximum number of iterations, the most widely used is cross-validation. This method is most effective in stopping training when the best generalization is achieved. It consists of separating a small part of the training data and using it to evaluate the trained network. Thus, training should stop the moment the network performance, quantified by the mean square error (MSE), begins to decrease or stagnates [18, 19, 20, 31].  
Lastly, in all MLP models created in this paper the activation function selected was the reLU function (rectified linear unit: \(\max(0,x)\)) [54], the kernel initializer was normal and the optimizing algorithm was the adaptive moment estimation (Adam) [55]. On the other hand, the batch size used to train the MLP neural networks was equal to 64 (mini-batch gradient descent [56]) and the patience (limit to stop the training if the performance of the model does not improve) was 100 epochs.  
Figure 1: Multi-layered perceptron (MLP) neural network architecture with \(N\) inputs, \(g\) hidden layers and univariate output.