## 4 Results and Discussion
### Evaluation of Experimental Results
#### 4.4.2 Evaluation of Fault Diagnosis Method  
First, we compared the performance of the proposed method with that of its variants. The confusion matrix of the multi-class problem was used, as shown in Figure 11. Each column of the matrix represents the instances in a predicted fault category, while each row represents the instances in an actual fault category. The effectiveness of diagnosing each type of fault was further quantified by the mentioned indicators: _accuracy, recall, precision, specificity_, and _F1-score._ In this section, _recall_ corresponds to the ability to identify a type of specific fault, _precision_ represents the degree of success of methods when a fault type is predicted, and _specificity_ corresponds to the capacity of methods to refuse the identification of a specific fault type. After obtaining the above indicators for each fault type, the overall classification performance was evaluated using the macro-average value of each indicator, which is calculated as:  
\[Ave-indicator=\frac{1}{Q}\sum_{i=1}^{Q}indicator_{Fi}, \tag{19}\]  
where \(Q\) is the number of fault categories, \(Fi\) is the label of the fault. In this paper, \(Q=7\). Table 6 shows the comparing results, which indicate that the proposed S-ECNN model is more effective than its variants in fault diagnosis. By collaboratively using the labeled and the unlabeled alarm sequences, all the indicators have been improved. Particularly, compared with the S-ECNN-rand model, the _ave-precision_ of the S-ECNN model increases by 13.1%, indicating that this method is more effective at fault diagnosing while avoiding false identification. Second, we compared the proposed method with a cluster analysis (CA) method [22], a multi-dimensional information processing (MIP) method [23], and a similarity analysis (SN) method [24]. The alarm data were preprocessed and the alarm sequences were segmented using the proposed method in this paper. In the CA method, the density-based spatial clustering of applications with noise was applied. In the MIP method, each alarm sequence was labeled with the most possible fault. When using the SN method, about 75% of the alarm sequences for each fault category were used to extract the feature vectors. It is noteworthy that these methods can only analyze the labeled alarm sequences, while the proposed method collaboratively uses the labeled and the unlabeled alarm sequences. For a fair comparison, the S-ECNN-rand model, which only uses the labeled alarm sequences, is also involved in the following comparison. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & _Ave-Accuracy_ & _Ave-Recall_ & _Ave-Precision_ & _Ave-Specificity_ & _Ave-F1\_Score_ \\ \hline S-ECNN-rand & 93.3\% & 75.0\% & 77.4\% & 96.0\% & 76.2\% \\ S-ECNN-static & 91.6\% & 73.8\% & 76.2\% & 95.2\% & 75.0\% \\ S-ECNN & 97.0\% & 89.3\% & 90.5\% & 98.3\% & 89.9\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The comparing indicators with the model’s variants for fault diagnosis. Figure 11: The confusion matrices for fault category prediction result. The mentioned macro-average indicators were used to quantify the effectiveness of these methods. For the SN method, the fault type of one unknown alarm sequence may not be assigned with a historical fault. In this situation, the FP in a confusion matrix cannot be deduced. Then the _ave-precision_, _ave-specificity_, and _ave-F1-score_ cannot be calculated. The comparing results are shown in Figure 12. As we can see, compared with the existing methods, the S-ECNN-rand model has achieved some success. The _ave-accuracy_, _ave-precision_, _ave-specificity_, and _ave-F1-score_ of the S-ECNN-rand model are higher than that of the existing methods, which proves the effectiveness for fault diagnosis of alarm sequences. However, the _ave-recall_ of the S-ECNN-rand model is lower than that of the SN method, which indicates that the ability to identify a type of specific case for the S-ECNN-rand model is not good enough. For the proposed method, all the indicators have been further improved.