# An EWS-LSTM-Based Deep Learning Early Warning System for Industrial Machine Fault Prediction
## 4 The Case Study Phases and Results
### The Deep Learning Approach Implementation  
In the previous section, it is discussed how data are distributed over time, which features are important to consider and how many errors need to be identified to prevent the machine from malfunctioning. This scenario suggests the need to address a multi-classification challenge, trying to predict the occurrence of an error as ahead of time as possible. To correctly classify the incoming data, a deep learning approach was chosen and adopted, as suggested in several recent studies [60,61]. Among the many neural networks available in the literature, the LSTM was selected as it is particularly well suited to handle the irregular and time-dependent data generated by industrial sensors. Additionally, to mitigate the problem of gradient vanishing, the attention mechanism to enhance the model's ability to focus on the most informative parts of the data [62] has been used. Therefore, the generation of the functional EWS is based on the exploration of various strategies, including the following:  
* Creating a single neural network able to multi-classify the five errors;
* Creating multiple neural networks, each specialised in a single error. To correctly justify the adopted approach, it was necessary to further investigate the distribution of the five error classes. The number of times a signal changed from \"0\" to any of the error classes was counted. The resulting count is reported in the row \"Total occurrences\" of Table 6.  
It can be observed that the total number of error events changes drastically from one class to another. The first strategy involved the usage of a single neural network to multi-classify the errors in a naturally unbalanced dataset with an unbalanced number of error classes. Additionally, this would result in the creation of a large model, both in size and computational complexity. Although the entire system is not meant to work with real-time requirements, this approach limits the extendability of the system, leading to the need for a single node with high computational capabilities. To adopt the second one, it is required to extract chunks of data from the dataset related to the specific error class and feed the models singularly. This would require that the dataset is passed to each individual neural network model and the use of a finalisation step to highlight the most probable error class (e.g., compare all network results and pick the one with the highest elicitation). Therefore, under the mentioned hypothesis, it was decided to adopt the second approach as the research team considered it more extensible and robust, being aware that some neural networks might perform better than others due to the different number of examples provided. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Error** & **2509** & **2556** & **2557** & **2558** & **2559** \\ \hline
**Total** **occurrences** & 56 & 14 & 41 & 12 & 37 \\ \hline
**Average TU** **number** & 93 & 234 & 151 & 63 & 125 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Training/testing dataset size for each error class. To extract a discrete number of TU rows with which each neural network would be trained and tested, a \(\Delta t\) around each error occurrence was considered. Specifically, all the features up to \(t=300\) [s] (5 min) before the error were taken, along with the first five TU elements following the occurrence. This extraction window was agreed with the domain experts as a result of the intrinsic machine behaviour during the paper blocking process (e.g., the slowing of the embosser rolls). Furthermore, since the whole dataset exhibits an imbalance between error and non-error classes, retaining only a portion around the error class can help to reduce overfitting. Each time frame segment encompassed a variable number of TUs including the machine's \"normal functioning\" and relevant feature values describing the error class.