# Stacked Ensemble of Recurrent Neural Networks for Predicting Turbocharger Remaining Useful Life
## 7 Experimental Results
In order to better visualise the training process of base models in comparison to the meta model, Figures 10 and 11 depict the best base model and meta model learning curve, respectively. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **MAE of TTE Predictions** \\ \hline Mean time to failure (Baseline) & 277.17 \\ \hline Best individual model & 139.32 \\ \hline Mean of all base models & 125.63 \\ \hline Median of all base models & 124.26 \\ \hline Stacked ensemble & 105.79 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Model comparison based on MAE of the TTE predictions. Figure 8: Comparing stacked ensemble results to the target values. Figure 9: Comparing best model results to the target values. Please note that for both learning curves early stopping is applied (early stopping epoch for best base model is 63 and for meta model is 43). The error on both validation and training sets is lower for meta model, except at the beginning of the curves for the training set. The reason for the error of the base model being lower at the beginning for the training set is that mini-batch training is used for base models, but incremental training is used for meta model. It can also be seen that the evolution of the meta model error is much more stable in comparison to the base model. As mentioned in Section 4 a sign of diversity in stacking is that the base models' predictions have low correlation with respect to each other. Table 6 depicts some statistics about correlation of base models. \begin{table}
\begin{tabular}{c c} \hline Mean correlation of all models & 0.56 \\ \hline Max correlation & 0.75 \\ \hline Min correlation & 0.42 \\ \hline \end{tabular}
\end{table} 
Table 6: Correlation statistics of base models. Figure 11: Learning curve of the meta model. Figure 10: Learning curve of the best base model. The average correlation of 0.56 is a good indicator of diversity between base models. Please note that in the context of stacking 0.75 correlation (the max correlation) is not still a high correlation given that the models are trying to predict the same target. The maximum correlation is for two LSTM models with the same architecture shown in Table 1 first architecture with same feature subsets but two different past dependency resolution. The lowest correlation is between an LSTM and CONV_LSTM shown in first architecture of Table 1, and second architecture of Table 2 respectively, on two different feature subsets and LSTM with past dependency resolution 5 and CONV_LSTM with past dependency resolution 20. This behaviour can be an indicator of maintaining diversity by looking at different aspects. In Figure 12 we present how the correlation between base models and the meta model is related to the performance of the former. It can be seen that for base models with between 0.5 and 0.6 correlation to the meta model, the MAE values are almost steady (with only minor fluctuations). After that, as the correlation increases the MAE reduces until the correlation reaches to almost 0.7. From this point on, the MAE values seem to remain steady again. Decrease in MAE as the correlation increases is expected since the meta model is predicting target values; hence, higher correlation between the base models and the meta model means higher predictability and consequently lower MAE.