## 6 Results and Evaluation  
### Results  
In summary, we first implemented a basic rule-based reasoning system, which was introduced in Section 4, for predicting the clash types according to the clash detection report generated by BIM software, and obtained the preliminary predictive accuracy rate of 60% (194/326). Next, we implemented six common classifiers (three individual classifiers and three ensemble learning classifiers) and conducted the same machine learning process twice using two training datasets: training dataset #1, derived from the clash detection report with clash types labeled by two human experts, and training dataset #2, derived from merging training dataset #1 and a feature of clash types predicted by the rule-based reasoning. For the experiment with training dataset #1, the best predictive performance from 30 tests among three individual classifiers is obtained by the DT classifier (f1-score is 0.87), whereas the best ensemble learning classifier is the one using the strategy of bagging (f1-score is 0.91). Among the three individual classifiers, the k-NN classifier obtained the lowest f1-score, indicating that the results of both SVM and DT classifiers are considerably robust and trustworthy. The DT classifier outperforms the SVM classifiers by 10% probably because the dataset is relatively small, and the number of labels in this study is four rather than two or three, where SVM could function better [28]. The information gain of DT classifiers usually tends to be larger for the label with more cases in training data, so it may benefit from the nature of the training data we selected. Among the three ensemble learning classifiers, there is no significant difference in their performance. Still, all of them outperform the three individual classifiers, proving the findings suggested by previous studies of machine learning [28], especially compared with the SVM and k-NN classifiers. The experiment results with the training dataset #1 are also compared with the previous work by [PERSON] and [PERSON] [6], where six machine learning classifiers were implemented to filter out irrelevant clashes. Their best predictive performance in terms of f1-score, obtained by both random forest classifier and a rule-based classifier, Jrip, was 0.74, lower than the average f1-score of the random forest classifier in our study (0.881). This outperformance does not necessarily indicate that our classifiers are better because their classifiers may suffer from underfitting due to the smaller training dataset (204 versus 326 cases) and the larger number of features (10 versus 6 features). The other reason could also be the difference in the nature of training data, as we mentioned in the comparison between DT and SVM classifiers. For the experiments with training dataset #2, the predictive performances of all three individual classifiers reach a favorable level of 0.91, while the level is 0.94 or higher for ensemble learning classifiers. The best performances for both individual and ensemble learning classifiers remain to be DT and bagging, respectively. The predictive performances of all the classifiers achieved improvements in terms of average f1-score, as shown in Figure 11. Taking a closer look at the comparisons, the average f1-scores of the SVM and k-NN classifiers significantly increased by approximately 15%-17% while the \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Clash Types**} & \multicolumn{2}{c}{**Individual Classifiers**} & \multicolumn{2}{c}{**Ensemble Learning Classifiers**} \\ \cline{2-7}  & **DT** & **k-NN** & **SVM** & **Voting** & **Bagging** & **Random Forest** \\ \hline Errors & 0.932 & 0.889 & 0.914 & 0.94 & 0.955 & 0.958 \\ Deliberate clashes & 0.970 & 0.959 & 0.958 & 0.97 & 0.972 & 0.966 \\ Pseudo clashes & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 \\ Unknown & 0.766 & 0.577 & 0.515 & 0.823 & 0.844 & 0.746 \\ \hline Average & 0.942 & 0.906 & 0.905 & 0.95 & 0.957 & 0.944 \\ \hline \hline \end{tabular}
\end{table}