# CNN-Based Damage Identification of Submerged Structure-Foundation System Using Vibration Data
## 5 Development of 1-D CNN Models
### 1-D CNN Model Using SVD Responses
#### 5.2.3 Evaluation of the 1-D CNN Model Using SVD Responses  
Figure 20(a) plots the damage classification results of the CNN model using the SVD databank for the trained noise level of 1%. There were five misclassifications in damage level D3 and two misclassifications in damage level D4. The accuracy of classification reached 95.62%. As the noise level increased up to 5% (Figure 20(b)), the accuracy decreased to 90%, with one to nine misclassifications in damage levels D1-D4. Figure 17: Illustration of noise contamination to SVD responses: (**a**) 5% noise; (**b**) 10% noise. Figure 18: Visualization of SVD responses for training and evaluation of the CNN model. Figure 20c shows the damage classification results of the CNN model using the SVD databank for the untrained noise level of 6%. Misclassifications appeared at all damage levels except for the intact level (D0). The overall accuracy of classification was 86.88%. As the noise level increased to 10% (see Figure 20d), the accuracy decreased to 71.25%. There were fewer than five misclassifications out of 20 data for each damage level, except for  
Figure 19: Loss values of the 1-D CNN model trained by SVD responses. Figure 20: Damage classification results of the 1-D CNN model using SVD databank: (**a**) trained 1% noise; (**b**) trained 5% noise; (**c**) untrained 6% noise; (**d**) untrained 10% noise. The red tick () denotes the correct level of inflicted damage. damage levels D3 and D4. The number of misclassifications was significantly smaller than the model trained using time-history responses at the same noise level. Figure 21 summarizes the accuracy metrics of the CNN model trained using SVD responses under the effect of noise. Accuracy, false positive rate, false negative rate, and F1-score were analyzed (see Figure 21a-d), and the results were compared to the accuracy metrics of the CNN model trained using time-history responses. The accuracy of the CNN model trained by SVD responses ranged from 91% to 96% for noise levels 0-5%. It slightly outperformed the CNN model trained using time-history responses, but the difference in accuracy between the two models was not significant. The performance divergence became clearer at higher noise levels (6-10%). At a noise level of 10%, the accuracy of the CNN model trained using SVD responses decreased to 71% compared to 53% using the time-history responses, suggesting that the CNN model trained with SVD responses exhibited improved robustness against untrained noise levels. For trained noise levels (0-5%), FPR values ranged from 4% to 11% and increased significantly to 20% at a noise level of 6% and continued to rise to 41% at a noise level of 10%. Similarly, FNR values remained below 12.5% for noise levels 0-5%, but increased significantly to a maximum of 47% at noise level 10%. In Figure 21d, the F1-score of the CNN model trained using SVD responses was above 0.9 for trained noise levels 0-5%. The F1-score rapidly declined for the untrained noise levels and dropped to 0.72 at a noise level of 10%. The F1-score indicated that the performance is not as strong as the CNN model trained using time-history responses. The \(p\)-value of the one-tailed t-test [24] was employed to evaluate the difference in performance of two approaches. A low \(p\)-value (typically \(\leq\) 0.05) indicates that the differences in performance are statistically significant. In Figure 21d, the \(p\)-value of the F1-score was greater than 0.05, indicating that the performance comparison based on the F1-score was not statistically significant. As shown in Figure 21a-c, the \(p\)-values of accuracy, FPR, and FNR were less than 0.05, indicating that using SVD data improved the CNN performance significantly. Figure 21: Noise effect on the 1-D CNN model trained using SVD responses compared to the model trained using time-history responses (grey lines): (**a**) accuracy; (**b**) false positive rate; (**c**) false negative rate; (**d**) F1-score.