# SOPRENE: Assessment of the Spanish Armada's Predictive Maintenance Tool for Naval Assets
## 5 Results
### Diesel Engine for Propulsion  
The development has been subject to a mostly qualitative evaluation of the developers themselves and the organization involved. Some of these ideas are collected in this subsection:  
* Prediction: Despite the large data pools available, the RPM filtering rules out a significant piece of data when the engine was off. Due to this, when a large grouping is carried out (for example, 1 data/week or 1 data/month), very few data results. As stated in Section 4.1, it disables the correct convergence of certain models, making each one of them suitable for a specific scenario. Table 2 compares the different techniques  
Figure 3: Example of system output for five failure modes (named 125, 44, 78, 18 and 82) sorted temporally. Each row depicts the evolution of the probability of failure along the time. The points where the failure probability reaches its peak are marked with white dots. and methods, collecting the mean squared errors by using different grouping modes and horizons. As it is possible to observe in the table, LSTM-based methods provide better performances with the lower degree of aggregation. Figure 4 depicts an example of prediction by using a recurrent neural network, where both sudden events and tendencies are correctly predicted. Please note that Figure 4 contains a prediction, while Figure 2 represents the reconstruction error in an autoencoder. On the other hand, Table 2 also illustrates that long-term behaviors are better estimated by simpler regression methods. The prediction system is very sensitive to both the grouping of the data and the sizes of the window and horizon. Furthermore, it has been found that, in general, there is a strong correlation between the variables that pass the selection process. A window size of approximately twice the forecast horizon has been found to be sufficient to achieve good results in most scenarios. * Anomaly detection: For this stage, two metrics have been used to qualitatively measure the performance of the model during an interval of four years: on the one hand, the anomalies detected by the model have been correlated with the worship's engine alarm system. Although, this system does not collect malfunctions but operative conditions, it is possible to observe an indirect relation between them (see Figure 5). On the other hand, the results have been analyzed by maintenance experts which focused on specific known events. The unsupervised trained autoencoder model has been tested and the results have been satisfactory. The model was able to detect most of these anomalies, as can be observed in Figure 6. The value of this graph is found in the coincidences between signals along the \(x\)-axis. It should also be noted that this process is very sensitive to parameterization and can be configured to allow the passage of more or less anomalies through the reconstruction threshold and, within these, select a greater or lesser number of variables involved by using the Elbow Method parameters. * Failure diagnose: Using artificial datasets based on the engine's design values (described in the Failure mode effects and criticality analysis, FMECA) has allowed us to build classification models that determine which failure modes are occurring. However, this theoretical behavior of the engine does not have to always correspond to reality, since its operation may vary with the use, replacement or repair of parts, etc. The training of diagnostic models depends directly on this generator and so it is necessary to build a sufficiently large and varied dataset.