# Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation
## 7 Results and Discussion
### Recommendation Performance  
Our experiments mainly focus on two research questions:  
* **RQ1:** Does incorporating knowledge-base in our model produce better recommendation performance? * **RQ2:** Which types of product knowledge are most useful for top-N recommendation? * **RQ3:** What is the computational efficiency of our model compared to other recommendation algorithms? To answer **RQ1**, we report the results of our model and the baseline methods in Table 2. As shown in Table 2, the deep models with rich auxiliary information (DeepCoNN, CKE, and JRL) perform better in general than the shallow methods (BPR, BPR-HFT, VBPR, TransRec) on most datasets, which is coherent with previous studies [24; 31; 32]. Among different neural baselines, JRL obtains the best performance in our experiments. It produced 80% or more improvements over the matrix factorization baselines and 10% or more over the other deep recommendation models. Overall, our model outperformed all the baseline models consistently and significantly. It obtained 5.6% NDCG improvement over the best baseline (i.e., JRL) on _CDs and Vinyl_, 78.16% on _Clothing_, 23.05% on _Cell Phones_, and 45.56% on _Beauty_. This shows that the proposed model can effectively incorporate product knowledge graph and is highly competitive for top-N recommendation. Figure 3 depicts the recommendation performance of our model and baseline methods with different embedding sizes on _CDs & Vinyl_ and _Beauty_ datasets. Observations on the other two datasets are similar. As shown in Figure 3, the recommendation methods based on shallow models (BPR, BPR-HFT, and VBPR) obtain the best NDCG when the embedding size is fairly small (from 10 to 100), and larger embedding sizes usually hurt the performance of these models. In contrast  
to the shallow models, the results of neural models (i.e., JRL, DeepCoNN, CKE, and our model) show positive correlations with the increase of embedding sizes. For example, the NDCG of the best baseline (JRL) and our model improves when the embedding size increases from 10 to 300, and remains stable afterwards. Overall, our model is robust to the variation of embedding sizes and consistently outperformed the baselines. In Table 2, we see that the CKE model did not perform as well as we expected. Although it has incorporated reviews, images and all other product knowledge described in this paper, the CKE model did not perform as well as JRL and our model. One possible reason is that CKE only considers heterogeneous information in the construction of item representations, but it does not directly leverage the information for user modeling. Another potential reason is that CKE separately constructs three latent spaces for text, image and other product knowledge, which makes it difficult for information from different types of data to propagate among the entities. Either way, this indicates that the embedding-based relation modeling of our model is a better way to incorporate structured knowledge for product recommendation. From the results we can also see that datasets of different density result in different performance in our model. In particular, denser datasets (_Cell Phone_ and _Beauty_) generally get better ranking performance than sparser datasets (_CD & Vinyl_ and _Clothing_) in our model, which means more sufficient information can help to learn better models in our algorithm. To answer **RQ2**, we experiment the performance of our model when using different relations. Because we eventually need to provide item recommendations for users, our approach would at least need the _Purchase_ relation to model the user purchase histories. As a result, we train and test our model built with only the _Purchase_ relation, as well as _Purchase_ plus one another relation separately. As shown in Table 3, the relative performance of our models built on different relations varies considerably on the four datasets, which makes it difficult to conclude which type of product knowledge is the globally most useful one.