# SumMER: Structural Summarization for RDF/S KGs
## 5 Evaluation
### Machine Learning for Schema Node Selection  
Next, we focus on exploiting multiple importance measures for selecting schema nodes. For evaluating the performance of our ML algorithms, we use Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and Median Absolute Error (MedAE) as they are commonly used for evaluating regression problems [29]. The \(\hat{y}\) is the predicted value, \(y\) is the corresponding actual value and samples is the number of samples:  
\[MAE(y,\hat{y}) =\frac{1}{n_{samples}}\sum\limits_{i=0}^{n_{samples}-1}\lvert y_{ i}-\hat{y}_{i}\rvert\] \[MSE(y,\hat{y}) =\frac{1}{n_{samples}}\sum\limits_{i=0}^{n_{samples}-1}(y_{i}- \hat{y}_{i})^{2}\] \[MedAE(y,\hat{y}) =median(\lvert y_{1}-\hat{y}_{1}\rvert,\ldots,\lvert y_{n}-\hat{y }_{n}\rvert)\]  
As we are only looking for the top-k nodes, we evaluate those metrics on the top-k nodes only. Further, we examine the performance of the ML algorithms on selecting the 10%, 15%, 20%, 25% and, 30% most important schema nodes out of the total schema nodes  
Figure 3: [PERSON]’s rank correlation for all datasets, for the various importance measures with the golden standard ranking.  
available in the dataset. We use the DBpedia v3.9 as the training dataset and the DBpedia v3.8 and SWDF as the test datasets.  
**DBpedia:** The results for DBpedia v3.8 are shown in Figure 4. In addition, Table 6 shows part of the confusion matrix of the various algorithms for the selection of the 10%, 15%, 20%, 25%, 30% top nodes. As shown in Figure 4, the Decision Tree regressor performs best in most of the cases, whereas most of the algorithms show a relatively good performance.  
Table 6 presents a part of the confusion matrix where we can see the true positives (TPs) that the various ML algorithms predict. As shown Decision Tree regressor makes the best predictions on the true positives (TP) in most of the cases (10%, 15%, 20%, 25%, 30%). Adaboost Regressor has also a good performance in selecting the TPs however it is not the best performer overall as seen also in Figure 4.  
**SWDF:** The results for SWDF are shown in Figure 5, whereas Table 6 shows part of the confusion matrix of the various algorithms for the selection of the 10%, 15%, 20%, 25%, 30% top nodes. In this experiment, the Adaboost, Gradient Tree Boosting and Decision Trees regressors perform best in terms of MAE, MSE and Median Absolute Error, whereas looking in Table 7 we can identify that Decision Tree Regressors performs also best in detecting the TPs. As the information provided in confusion matrices is only partial it is natural to see differences with the regression evaluation measures (MAE, MSE, Median Absolute Error)--regression measures compute the distance error for all the values while the confusion matrix computes only to miss or hit for the class.  
Overall, we can easily see that the Decision Trees Regressor shows an excellent performance among the two datasets and can always make good predictions. As such for the rest of the evaluation we only include the Decision Tree Regressor. Finally, we have to note that the highest-ranked features from the Decision Tree regressor are Hits, PageRank and the number of instances.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **10\%** & **15\%** & **20\%** & **25\%** & **30\%** \\ \hline AdaBoost Regressor & 15/35 & 23/53 & 30/70 & 50/88 & 78/106 \\ Gradient Boosting Regressor & 13/35 & 22/53 & 36/70 & 47/88 & 69/106 \\ Extra Tree Regressor & 14/35 & 22/53 & 42/70 & 54/88 & 64/106 \\ Random Forest Regressor & 15/35 & 39/53 & 51/70 & 65/88 & 77/106 \\ Decision Tree Regressor & 14/35 & 22/53 & 46/70 & 69/88 & 85/106 \\ Bayesian Ridge & 13/35 & 22/53 & 33/70 & 47/88 & 65/106 \\ Elastic Net & 13/35 & 21/53 & 32/70 & 47/88 & 64/106 \\ \hline \hline \end{tabular}
\end{table}
Table 6: TP for test results of DBpedia v3.8 (Predicted Nodes/Actual nodes).  
Figure 4: Evaluation Measures for each algorithm for DBpedia v3.8.