# Personalized Chinese Tourism Recommendation Algorithm Based on Knowledge Graph
## 4 Experiment
### Evaluation Metrics  
In this paper, the performance evaluation of the Bi-LSTM+CRF algorithm mainly adopts the accuracy rate (_Precision_), the recall rate (_Recall_) and the comprehensive index F1 value. The prediction and the recall are defined as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{12}\]  
\[Recall=\frac{TP}{TP+FN} \tag{13}\]  
In the formula, \(TP\) is the number of samples that are positive and the prediction is also positive, \(FP\) is the number of samples that are negative and the prediction is positive, and \(FN\) is the number of samples that are positive and the prediction is also negative [30,31]. The \(F1\) value is the weighted average of recall and precision. Recall and precision indicators may be contradictory in some application scenarios. The \(F1\) value is an evaluation indicator that combines the two indicators, which comprehensively reflects the overall situation. The formula is as follows:  
\[F1=2\cdot\frac{Recall\times Precision}{Recall+Precision} \tag{14}\]  
In this paper, link prediction is used to evaluate the training efficiency of the knowledge representation model. Link prediction predicts the missing head entity or tail entity in the correct triplet. In this scenario, for each missing entity, we use a ranking method to find its candidate entities from the knowledge graph of tourist attractions, sort the candidate entity, and then judge the training effect based on the ranking of the missing entities among the candidate entities. For a triplet \((h,r,t)\), given \((h,r)\) to predict the tail entity \(t\), and given \((r,t)\) to predict the head entity \(h\), we use Formula (2) to calculate the predicted triple score, and sort the scores in ascending order, two indicators are used to measure the quality of model training, the Mean Rank and the top 10 Hit rate (Hits@10). The Mean Rank represents the average rank of all correct entities in the test set. The value is the lower, the rank of the entity is the higher and the effect of model training is the better. The top 10 hit rate indicates the ratio of the number of correct entities in the top 10 to the total number of entities. Hits@10 is the higher, and the training performance of the model is the better. Consider that the user may choose the attraction to be in the first few items in the recommended list in practical applications. Therefore, this paper uses the evaluation indicators commonly used in the recommendation system: one is to calculate the Mean Absolute Error (MAE) based on the user's true rating of the attraction to determine the \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Entity**} & \multicolumn{2}{c}{**Attribute**} \\ \hline \multirow{2}{*}{Attraction} & Natural & Cultural & Location of & \multirow{2}{*}{Scoring} & \multirow{2}{*}{In the news} \\  & landscape & landscape & attractions & & \\
9100 & 6200 & 2900 & 1100 & 9100 & 27,300 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The data statistics of knowledge graph. accuracy of the predicted attraction rating, and the other is to judge the recommendation effect based on the hit rate. The mean absolute error _MAE_ is a measure of the error between two values [32]. The user's rating and the attraction rating predicted by the recommendation algorithm are incorporated into the calculation formula, and then the average value of the error of all of the predicted score attractions is calculated. The accuracy of the prediction depends on the average error. The smaller the _MEA_ value is the better; the formula is as follows:  
\[E_{MAE}=\frac{1}{n}\sum_{i\in U,j\in I}|p_{i,j}-r_{ij}| \tag{15}\]  
In the formula, the set \(U\) and the set \(I\) are the user set and the set of attractions, respectively. \(P_{ij}\) is the evaluation score of the attraction \(j\) by the user \(i\), \(r_{ij}\) is the true score of the attraction \(j\) by the user \(i\), and the number of \(p_{ij}\) is \(n\).