# An EWS-LSTM-Based Deep Learning Early Warning System for Industrial Machine Fault Prediction
## 4 The Case Study Phases and Results
### The Performance Measurement  
Each neural network was independently trained 10 times, according to the parameters specified in Section 4.4, and in this section, the best results among those runs are presented. For each model, the confusion matrix (along with accuracy), precision, recall, and the area under the receiver operating characteristic (AUROC) were calculated using the test set extracted prior to the training phase. Following the same approach, the retrieved results are reported in Tables 7-11. In particular, each row represents the specified metrics for the prediction time at the given \(TU+1+\cdots+10\). \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **AUROC** \\ \hline \(t+1\) & 0.89 & 0.06 & 0.45 & 0.41 \\ \(t+2\) & 0.89 & 0.07 & 0.48 & 0.45 \\ \(t+3\) & 0.89 & 0.06 & 0.42 & 0.36 \\ \(t+4\) & 0.89 & 0.08 & 0.70 & 0.45 \\ \(t+5\) & 0.89 & 0.08 & 0.52 & 0.43 \\ \(t+6\) & 0.89 & 0.07 & 0.52 & 0.44 \\ \(t+7\) & 0.88 & 0.07 & 0.54 & 0.42 \\ \(t+8\) & 0.88 & 0.08 & 0.66 & 0.44 \\ \(t+9\) & 0.88 & 0.07 & 0.46 & 0.37 \\ \(t+10\) & 0.87 & 0.09 & 0.59 & 0.47 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Metric performances on the test set for the model 2509. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **AUROC** \\ \hline \(t+1\) & 0.97 & 0.02 & 0.58 & 0.60 \\ \(t+2\) & 0.97 & 0.16 & 0.92 & 0.89 \\ \(t+3\) & 0.97 & 0.00 & 0.00 & 0.04 \\ \(t+4\) & 0.96 & 0.03 & 1.00 & 0.76 \\ \(t+5\) & 0.96 & 0.08 & 0.46 & 0.42 \\ \(t+6\) & 0.96 & 0.09 & 0.93 & 0.94 \\ \(t+7\) & 0.96 & 0.04 & 1.00 & 0.93 \\ \(t+8\) & 0.95 & 0.03 & 0.86 & 0.76 \\ \(t+9\) & 0.94 & 0.09 & 0.71 & 0.74 \\ \(t+10\) & 0.94 & 0.03 & 0.33 & 0.40 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Metric performances on the test set for Model 2556. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **AUROC** \\ \hline \(t+1\) & 0.95 & 0.09 & 0.92 & 0.83 \\ \(t+2\) & 0.93 & 0.35 & 0.67 & 0.68 \\ \(t+3\) & 0.92 & 0.41 & 0.58 & 0.59 \\ \(t+4\) & 0.91 & 0.07 & 0.67 & 0.65 \\ \(t+5\) & 0.90 & 0.40 & 0.77 & 0.75 \\ \(t+6\) & 0.90 & 0.22 & 0.64 & 0.64 \\ \(t+7\) & 0.90 & 0.10 & 0.73 & 0.71 \\ \(t+8\) & 0.89 & 0.44 & 0.86 & 0.83 \\ \(t+9\)Model 2556 consistently demonstrates superior accuracy performance across all forecast horizons, indicative of its robust predictive capabilities. Models 2509 and 2557 share similar accuracy trends, whereas Models 2558 and 2559 exhibit slightly diminished but still commendable accuracy levels. Precision analysis reveals that Model 2556 achieves notable precision values, particularly at \(t+2\) and \(t+3\), underscoring its proficiency in accurately identifying positive instances. In contrast, Model 2557 displays variability in precision across forecast horizons. Models 2509, 2558, and 2559 manifest lower precision values, which implies a higher incidence of false positives. In terms of recall, Model 2556 performs consistently well, attaining elevated recall values. Model 2559 shows sustained recall values, reflecting an effective equilibrium between false negatives and true positives. Models 2509, 2557, and 2558 display disparate recall outcomes across forecast horizons. Regarding discriminatory power, Model 2556 maintains consistently high AUROC values, emphasising its ability to distinguish between classes. Model 2559 also demonstrates commendable performance in AUROC. Models 2509, 2557, and 2558 exhibit variability in AUROC values on different forecast horizons. Additional considerations can be made by relating these results to the training data presented in Table 6. The superior performance of Model 2556 on various metrics aligns with its relatively large training dataset size (14 chunks with an average of 234 entries per chunk). The abundance of data may have contributed to the robust learning and generalisation capabilities of this model.