## 3 Steel Truss Experiment
### Signal Recognition and Analysis
#### 3.3.3 Analysis of Results and Discussion  
To validate the effectiveness of the model obtained after the aforementioned training, a test set was used to evaluate the model. The construction method of the test set is shown in Table 2. This paper employs four metrics: Confusion Matrix, Precision, Recall, and F1 score, to measure the classification accuracy of the model on the test samples. The Confusion Matrix, typically represented as an n \(\times\) n matrix, is a standard format for accuracy evaluation, also known as an error matrix. The definition of accuracy is provided in Formula 14, while the definitions and calculation methods of Precision, Recall, and F1 score are presented in Table 4. The final analysis results are shown in Figure 10 and Table 5.  
In the classification results in Figure 10, it can be seen that for the 960 samples in the test set, the overall recognition accuracy of the 15-dimensional combined features reached 98.75%. Specifically, among the one hundred and twenty samples in each category, except for four samples in both the third and sixth categories being misclassified, the other categories can basically be accurately identified. Among them, only one sample in the second, fourth, fifth, and eighth categories was misclassified, while the samples in the first  
Figure 9: The influence of feature dimension on accuracy and training time. and seventh categories were fully recognized without any misclassification. This indicates that the classifier can correctly identify the sample categories in most cases. In addition, as can be seen from the recognition results in Table 5, the precision is above 0.98 for all categories except for the slightly lower precision of 0.97 in the fourth category. Notably, the precision for the seventh category of bolt loosening sound signals reaches 1, indicating that all samples predicted as the seventh category are truly from the seventh category. The recall rates are high for all categories except for the slightly lower rates of below 0.99 in the third and sixth categories. Specifically, the recall rates for the first, seventh, and eighth categories all reach 1, meaning that all true samples from these categories are correctly predicted by the model. The F1 score are above 0.98 for all categories, except for a slightly lower score of 0.97 in the sixth category. Notably, the F1 score for the first, fifth, and seventh categories all reach 1, indicating excellent performance of the model in these categories. Meanwhile, in this paper, in addition to using combined features for training recognition, independent training recognition using 24-dimensional MFCC features and 16-dimensional WPES features is also attempted separately. The specific data on recognition accuracy, precision, recall, and F1 score are shown in Figure 11 and Table 6. As can be clearly seen from Figure 11, the recognition accuracy using only MFCC features is 96.2%, while the recognition accuracy using only WPES features is 97.7%. Both of these accuracies \begin{table}
\begin{tabular}{c c c} \hline \hline
**Indicator** & **Formula** & **Meaning of Indicator** \\ \hline Precision & \(\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}\) & The proportion of correct predictions among all positive predictions made by the model \\ Recall & \(\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}\) & The proportion of positive predictions made correctly by the model among all true positives \\ F1 score & \(\text{F1}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}\) & The harmonic mean of precision and recall \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Precision, Recall, and F1 score. Figure 10: Confusion matrix. are lower than the accuracy achieved by using the 15-dimensional combined features for recognition, which reaches a high level of 98.8% on the test set.