# Learning Knowledge Using Frequent Subgraph Mining from Ontology Graph Data
## 3 Results
### Evaluation Metrics and Methods  
The goal of these experiments is to predict the movie ratings and therefore accuracy was used as the basic evaluation metric. On the other side, considering that other methodologies used for performance comparison were designed to provide recommendations, the precision, recall, and F1-score were compared. Moreover, considering that precision, recall, and F1-score are typically evaluation indicators for binary classes and that this experiment aims to predict for multi-class, this study used macro average precision, macro average recall, and macro F1-score as the evaluation metrics. These are similar to the above three metrics and are known to be suitable metrics for multi-class classification. The equations used to calculate them are shown below [24].  
\[Accuracy=\frac{\sum_{k=1}^{K}(\text{the number of movies with identical actual and predicted rate})}{\text{the number of movies in test}} \tag{10}\]  
\[K:\text{the number of rates}(\text{i.e. }\ref{eqn:recision}\text{ in here})\]  
\[Precision_{k}=\frac{TP_{k}}{TP_{k}+FP_{k}} \tag{11}\]  
\[Recall_{k}=\frac{TP_{k}}{TP_{k}+FN_{k}} \tag{12}\]  
\[MacroAveragePrecision=\frac{Precision_{k}}{K} \tag{13}\]  
\[MacroAverageRecall=\frac{Recall_{k}}{K} \tag{14}\]  
\[Macro\ F1-Score=2\times\left(\frac{MacroAveragePrecision\times MacroAverageRecall }{MacroAveragePrecision+MacroAverageRecall}\right) \tag{15}\]  
Equations (11) and (12) are calculated for each rating, and the other equations are calculated for each user. The existing methods considered for comparison in this study are kNN [20], SVD [21], SVD++ [20], and NMF [22], which have been often used for comparison in recent recommendation studies [16; 25]. As these methods utilize only the user and rating information to generate a numerical prediction, for the sake of comparison,  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Property**} & \multicolumn{2}{c}{**Experiment 1**} & \multicolumn{2}{c}{**Experiment 2**} & \multicolumn{2}{c}{**Experiment 3**} \\ \cline{2-7}  & **Training** & **Test** & **Training** & **Test** & **Training** & **Test** \\ \hline MovieCompany & 140,654 & 36,222 & 140,759 & 36,117 & 141,128 & 35,748 \\ MovieCountry & 68,585 & 17,663 & 68,691 & 17,557 & 68,754 & 17,494 \\ MovieGenre & 143,655 & 36,810 & 143,947 & 36,518 & 143,688 & 36,777 \\ MovieKeyword & 588,456 & 151,113 & 588,580 & 150,989 & 588,765 & 150,804 \\ MoviePopularity & 52,833 & 13,514 & 52,838 & 13,509 & 52,839 & 13,508 \\ MovieRating & 52,867 & 13,516 & 52,867 & 13,516 & 52,867 & 13,516 \\ MovieReverune & 45,396 & 11,686 & 45,449 & 11,633 & 45,493 & 11,589 \\ MovieRuntime & 52,823 & 13,513 & 52,831 & 13,505 & 52,829 & 13,507 \\ MovieVote\_Average & 52,833 & 13,514 & 52,838 & 13,509 & 52,839 & 13,508 \\ MovieVote\_Count & 52,833 & 13,514 & 52,838 & 13,509 & 52,839 & 13,508 \\ Production & 365,444 & 94,416 & 366,116 & 93,744 & 366,862 & 92,998 \\ Sound & 273,865 & 70,332 & 274,290 & 69,907 & 273,992 & 70,205 \\ UserWatching & 52,867 & 13,516 & 52,867 & 13,516 & 52,867 & 13,516 \\ VisualEffects & 121,357 & 31,731 & 121,798 & 31,290 & 121,958 & 31,130 \\ WatchingMovie & 52,867 & 13,516 & 52,867 & 13,516 & 52,867 & 13,516 \\ Writing & 132,087 & 33,992 & 132,281 & 33,798 & 132,487 & 33,592 \\ \hline Total & 5,025,103 & 1,298,186 & 5,039,700 & 1,283,589 & 5,039,134 & 1,284,155 \\ \hline \hline \end{tabular}
\end{table}
Table 7: \(Cont.\)the prediction result was rounded up to the nearest 0.5 (the rating unit) and assigned to the corresponding rating.