# A Novel Integrated Fault Diagnosis Method Based on Digital Twins
## 4 Experiments and Results
### Comparative Experiments
#### 4.2.3 Evaluation Metrics  
In this paper, we employed five metrics to evaluate the performance of each classification model: prediction accuracy (PA), Noise Suppression (NS), Precision (PC), Recall (RC), and F1 score. This section will provide a detailed explanation of these metrics.  
**Definition 1**.: _Let \(n\) denote the number of samples correctly predicted by the model, and \(m\) denote the total number of samples predicted by the model. Therefore, we define the prediction accuracy (PA) of the model as_  
\[PA=\frac{n}{m}\times 100\%. \tag{13}\]  
PA reflects the prediction accuracy of the model. A higher PA value indicates a higher prediction accuracy, signifying better model performance.  
**Definition 2**.: _We introduce additive Gaussian white noise into the original dataset to test the model's performance under noise interference [29]. Let \(PA_{normal}\) be the accuracy obtained by the model under normal conditions, and \(PA_{noise}\) be the accuracy obtained by the model after adding noise. Therefore, we define the model's Noise Suppression (NS) as_  
\[NS=\frac{PA_{normal}-PA_{noise}}{PA_{normal}}\times 100\%. \tag{14}\]  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Layer** & **Name of Layer** & **Number of Layer** & **Output Shape** & **Number of Parameters** \\ \hline \multirow{6}{*}{DCBA} & Layer1 & Conv1d & 1 & (None, 32, 10) & 3680 \\  & Layer2 & BLSTM & 3 & (None, 10, 256) & 956,416 \\  & Layer3 & Attention & 1 & (None, 256) & 256 \\  & Layer4 & Linear & 1 & (None, 10, 256) & 65792 \\ \cline{1-1}  & & Dropout & 1 & (None, 256) & - \\ \cline{1-1}  & & Linear & 1 & (None, 8) & 2056 \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Cont._NS reflects the model's ability to maintain prediction accuracy in the presence of noise interference. The closer the NS value is to 1, the less sensitive the model is to noise, indicating stronger noise resistance and better model performance.  
**Definition 3**.: _Let TP be the number of true positive samples correctly predicted by the model, and FP be the number of false positive samples incorrectly predicted as positive by the model. Therefore, we define the model's Precision (PC) as_  
\[Precision=\frac{TP}{TP+FP}. \tag{15}\]  
Precision reflects the accuracy of the model in identifying positive instances. Specifically, a higher Precision indicates that a higher proportion of the samples predicted as positive by the model are indeed true positives.  
**Definition 4**.: _Let TP be the number of true positive samples correctly predicted by the model, and FN be the number of false negative samples incorrectly predicted as negative by the model. Therefore, we define the model's Recall (RC) as_  
\[Recall=\frac{TP}{TP+FN}. \tag{16}\]  
Recall reflects the model's ability to identify actual positive samples. Specifically, a higher Recall indicates a stronger ability of the model to identify positive samples.  
**Definition 5**.: _The F1 score is the harmonic mean of Precision and Recall. Precision reflects the proportion of true positives among the samples predicted as positive by the model, while Recall reflects the proportion of true positives identified among all actual positive samples. Therefore, the F1 score can be calculated as follows:_  
\[F1\ Score=2\times\frac{Precision\times Recall}{Precision+Recall}. \tag{17}\]  
The F1 score reflects the overall performance of the model in classification tasks. Only when both Precision and Recall are high will the F1 score be high. Therefore, a higher F1 score indicates better balance and superior performance of the model.