## 4 BIM Element Classification
### Results and Discussion
#### 4.3.3 Comparison of Classification Results on Mixed Data Sets  
In order to further verify the feasibility and effectiveness of machine learning algorithms in BIM element classification, two mixed data sets (data3 and data4) composed of three types of IFC data were tested. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{2}{c}{**ON(2048)**} & \multicolumn{2}{c}{**NI(13176)**} & \multicolumn{2}{c}{**EL(1932)**} \\ \cline{2-6}  & **Training** & **Test** & **Training** & **Test** & **Training** & **Test** \\ \hline SVM & 0.9988 & 0.9908 & 0.9461 & 0.9453 & 0.9784 & 0.9767 \\ ANN & 1 & 0.9816 & 0.9456 & 0.9450 & 0.9672 & 0.9587 \\ DT & 1 & 0.9908 & 0.9461 & 0.9454 & 0.9784 & 0.9767 \\ RF & 1 & 1 & 0.9461 & 0.9454 & 0.9784 & 0.9767 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Optimal classification results for a single different dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Optimization** & **SVM** & **ANN** & **Decision Tree** & **Random Forest** \\ \hline Training & 0.9988 & 1 & 1 & 1 \\ Accuracy & 0.9908 & 0.9816 & 0.9908 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Optimal classification results. In order to avoid an inter-class imbalance, resampling was carried out on 18,005 elements of data3, so that the number of elements of each type reached 11,692. At this point, the data set contained 46,768 elements. The data set was divided into training set, verification set and test set in the ratio 6:2:2, and BIM element classification was carried out on the data set by SVM, ANN, decision tree and support vector machine, respectively. The results obtained are shown in Table 11. The optimization of hyperparameters was performed using the grid-based search method. As can be seen from Table 11, the training and test accuracies for the four methods in Data3 hover around a value of 0.82, which shows a significant decrease compared with the performance for a single data set. The obfuscating matrix for the four methods above Data3, shown in Figure 12, explains the decrease in accuracy. Figure 12: Confusion matrix about Data3. \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & \multicolumn{1}{c}{**SVM**} & \multicolumn{1}{c}{**ANN**} & \multicolumn{1}{c}{**DT**} & \multicolumn{1}{c}{**RF**} \\ \hline Hyperparameter & \(C=20,\sigma=50\) & \(r=0.01,\alpha=0.99\) & DEFAULT & \(n=10\) \\ Train Accuracy & 0.8255 & 0.8221 & 0.8252 & 0.8252 \\ Test Accuracy & 0.8224 & 0.8215 & 0.8241 & 0.8241 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Data3 optimal classification results. It can be seen from Figure 12 that the confusion matrices for the SVM, decision tree and random forest methods are all the same, and the difference with ANN lies in the classification of elements such as MEMBER, which is only 0.03. This indicates that the main influence on the classification results is not that of the machine learning method itself, but rather the data set. By comparing the confusion matrices of the four methods, it can be seen that the element affecting the final classification accuracy is mainly MEMBER. An element that should be classified as MEMBER is misclassified as PLATE in 62% of the cases. By looking at the original information in Data3, we can see that the MEMBER element misclassified is consistent with PLATE in terms of features. Therefore, in order to improve the final classification accuracy, more features need to be mined in order to properly distinguish between MEMBER and PLATE. An additional factor (that cannot be ignored) is the three data sets by ON + TH + FO + NI + EL, i.e., the merging of five IFC files to extract information; the TH and FO contain only three elements for classification (excluding BEAM), and the number of elements in 11,396 accounts for more than 60% of the total elements in the collection data. As a result, these two IFC files of information changed the original data distribution. This is the main reason why the efficiency of the machine learning method when applied to the sets of data3 was less prominent.