## 4 Experimental Design and Results
### Findings and Analysis from the Experiments
#### 4.3.3 Confusion Matrix  
The confusion matrix is a tabular representation commonly employed to assess the performance of a classification model, illustrating the correlation between the model's predictions and the actual outcomes on a test dataset [38]. It serves as a valuable tool in classification problems, providing a detailed evaluation of the model's behavior and performance.  
The confusion matrix is presented as a two-dimensional table, with rows indicating the actual classes and columns representing the predicted classes made by the model. Table 1 illustrates an exemplar confusion matrix.  
In the confusion matrix, TP represents the correct prediction of positive samples as positive, indicating that the model correctly identifies the true positives. This means that the model successfully classifies positive instances as positive. TN represents the correct prediction of negative samples as negative, indicating that the model correctly identifies the true negatives. This means that the model successfully classifies negative instances as negative. FP represents the model's incorrect prediction of negative samples as positive, also known as Type I Error, indicating that the model incorrectly identifies actual negatives as positives. FN represents the model's incorrect prediction of positive samples as negative, also known as Type II Error, indicating that the model incorrectly identifies actual positives as negatives.  
Figure 11 displays the confusion matrix for the test dataset, providing a clear depiction of the classification accuracy performance for each class. The matrix reveals that the likelihood of misclassification between classes is minimal, indicating a model with excellent classification capabilities.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & & \multicolumn{2}{c}{**Predicted Results**} \\ \cline{3-4}  & & **Positive Example** & **Negative Example** \\ \hline \multirow{2}{*}{Real results} & Positive example & TP & FN \\  & Negative example & FP & TN \\ \hline \hline \end{tabular}
\end{table}
Table 1: Confusion matrix.  
Figure 10: ROC curves for each category.  
It can be observed that there are some incorrect predictions in the classes of Normal Operation, Fluid Pound, Gas Interference, and Delayed Closing of Traveling Valve. We have separately plotted the confusion matrices between these classes, as shown in Figure 12.  
From Figure 12, the model demonstrates a classification accuracy exceeding 0.94 for the classes of Normal Operation, Fluid Pound, Gas Interference, and Delayed Closing of Traveling Valve. This indicates that the Swin Transformer model is capable of accurately distinguishing similar waveform patterns under different working conditions, and it exhibits excellent performance.  
When we selected some paragraph cards and compared them, as shown in Figure 13, we found that there were some incorrect predictions. Particularly notable is that in the classes of Normal Operation, Fluid Pound, Gas Interference, and Delayed Closing of Traveling Valve, the waveform patterns of the phagraph cards may exhibit similarities, especially under certain specific operating conditions. This makes it challenging for the model to accurately differentiate between different classes in these similar waveforms.  
The phagraph card data may contain some noise, which could interfere with the model's learning process. This noise can lead to erroneous biases in the model's predictions for similar waveforms, consequently affecting its classification performance.  
Figure 11: Confusion matrix of test set. ((a) Normal Operation; (b) Fluid Pound; (c) Gas Interference; (d) Gas Locked Pump; (e) Delayed Closing of Traveling Valve; (f) Pump Barrel Split; (g) Fluid Pound and Delayed Closing of Traveling Valve).  
Figure 12: Confusion matrix for different working conditions. ((a) Normal Operation; (b) Fluid Pound; (c) Gas Interference; (e) Delayed Closing of Traveling Valve).