## 5 Results

Table 3: Comparison of the performances of the base models and ensemble. The bold face highlights the more efficient model. measurement. A dropout regularization that switches off 5% of neurons was also included to avoid overfitting problems. Finally, a dense layer was included to allow connection to the output of the neural network. The Adam optimizer, the most widely applied of its kind, was used to compile the neural network model. During compilation, the learning and optimization processes were conducted following the same criteria used in the regression models. The learning process was performed for each epoch, and the validation error was calculated. This process was repeated until the validation loss assumed a stable value, as presented in Figure 6a. The model results for the test set are as follows: MSE, 1.749; R-squared, 0.984; RMSE, 1.314; and MAE, 0.937. These results show that the selected model makes predictions with a high coefficient of determination. The MSE value is lower than the results obtained during the validation process, and the RMSE and MAE present similar values. In addition to these four metrics, the accuracy of the predictions was also graphically evaluated by comparing the models' results with the measured values for the two target variables: the bearing temperatures on the pump side (TEMP.MOTOR) (see Figure 6b) and the free side (TEMP.FREE). These results show a prediction quality much higher than that of the pure regression models. The residual values of the LSTM predictions tend to lie much closer to zero than the results of the regression model (see Figure 7a), indicating that the distance between the actual value and the value predicted by the model is much smaller in the case of a neural network. However, one must consider that both models work differently. While the regression model seeks to obtain the value of a feature through correlation with the rest of the variables, the LSTM neural network receives as an input the values of the five previous measurements to predict the temperature value in the next time step. It is also important to note that bearing temperature predictions were made for the value over the next 10 min. However, it is possible to train the neural network to estimate bearing temperature values for longer time intervals (see Figure 7b). Learning curves are among the most widely used tools for evaluating the performance of regression models. learning curves show how a model 'generalizes' for unseen data [72; 73; 74]. The performance pertaining to the learning curve in Figure 8a shows that the increase in the number of data points improves the model's learning process with respect to the validation set. The MSE of the validation set reduced not only with the increase in training size but also its standard deviation. Therefore, the learning curve shows the importance of the data points with respect to improving accuracy. It is also important to note that, in our case, from 6k data points onward, the MSE value becomes much more stable. Figure 6: Behavior of the trained LSTM model. (**a**) Evolution of the validation loss for the LSTM model. (**b**) Prediction for the TEMP.MOTOR variable (test set). Validation curves are beneficial for addressing under- and overfitting [64]. Here, instead of calculating the scores of the validation and training sets for different training set sizes (learning curves), the value of both scores is obtained from the variation in a model parameter. In Figure 8b, the ElasticNet model parameter \(\alpha\) is analyzed. The results confirm that the absolute value of the MSE for the validation set reaches the efficient value when \(\alpha=0.5\). In fact, the MSE value for the training set does not show appreciable variations. Consequently, there are two clearly differentiated trends in the graph. For alpha values less than 0.5, the effect of the penalty is low, so the model cannot generalize correctly. In other words, overfitting occurs.