## 7 Stage 2: ML Recognition
### Task 3: Local Damage Pattern Recognition  
From the two validation experiments in Tasks 1 and 2, it was shown that the ARIMA coefficients could accurately distinguish between damaged and undamaged states both globally and locally. However, in the present experiment with increased difficulty of the task, i.e., more classes to be classified, Table 7 shows that the performance of all models slightly decreased, especially in the complex models with RF and LR classifiers. In general, EVC provided the best accuracy of about 80% in all cases. ARI(7,1) with EVC again presented the best performance with the highest accuracy, i.e., 81.5% and 82.1% in floors 1 and 2, respectively. Although the results were lower than those in Tasks 1 and 2, they were much higher than a random guess of 33% for Task 3 three-class classification.  
The confusion matrix analysis, Figure 12b-d, indicated that the best classifier was more accurate in distinguishing between undamaged and damaged states but less accurate to identity the differences between damage levels (PL and AL). Moreover, Figure 12c,d shows that the model was prone to predict PL as being AL for the braces. This misclassification is explained by a coupling effect from different floors, where the nonlinear response of a single floor is influenced by the adjacent floors [14].  
Taking test record EQ12 as an example, there was only PL in floors 1 and 2 but AL in floor 3 during EQ12, Table 1. We inferred that the nonlinear response of floor 2 was influenced by both floor 3 with severe damage (AL) and floor 1 with a similar damage level (PL). Thus, using the ARIMA model with such simple structural idealization (each floor treated independently) may not render a very accurate damage pattern recognition task. Future work should make use of more accurate substructure idealization of the tested structural system [14].  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Floor \#** & **SVM** & **NN** & **RF** & **LR** & **EVC** \\ \hline \multirow{4}{*}{ARI(7,1)} & 1 & 96.7 & 96.7 & 93.4 & 96.4 & 96.7 \\ \cline{2-7}  & 2 & 97.5 & 97.2 & 96.2 & 97.4 & 97.7 \\ \cline{2-7}  & 3 & 98.5 & 98.7 & 98.0 & 98.5 & 98.7 \\ \hline \multirow{4}{*}{ARIMA(6,1,6)} & 1 & 95.3 & 95.7 & 88.4 & 93.4 & 95.8 \\ \cline{2-7}  & 2 & 96.2 & 96.0 & 92.3 & 94.5 & 96.4 \\ \cline{2-7}  & 3 & 97.8 & 97.6 & 93.9 & 97.6 & 98.0 \\ \hline \multirow{4}{*}{ARIMA(7,1,6)} & 1 & 95.8 & 95.3 & 88.7 & 92.9 & 95.8 \\ \cline{2-7}  & 2 & 96.9 & 95.9 & 91.9 & 93.5 & 96.9 \\ \cline{2-7}  & 3 & 97.6 & 97.8 & 94.1 & 97.0 & 98.0 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Accuracy (%) of the local damage detection for different candidate models.  
Further evaluation of the the test accuracy of floor damage pattern from a holistic point of view for a single event, i.e., predicting labels for a certain earthquake, can be based on majority labels predicted among the TS segments of each floor. Subsequently, normalizing the predictions for each label, a probability distribution is obtained, and the final event prediction is based on the highest probability. The predictions from a single test earthquake using ARI(7,1) are presented in Figure 13, and the final predictions were consistent with the true labels in the Table 1.  
Similar to the observations in Figure 12c,d, the model can easily distinguish between undamaged and damage states (EQ4 and EQ7) with high confidence. On the contrary, the difference between PL and AL was less significant (EQ12 and EQ15) but still acceptable. Therefore, the classification performance achieved in damage pattern recognition is also applicable in practice, especially from a holistic point of view for a single earthquake event regarding the typically difficult task of _damage pattern prediction_.