# An Ontology-Based Recommender System with an Application to the Star Trek Television Franchise
## 3 Experimental Validation
### Results Discussion  
Let us first discuss the results for the Warm System scenario. The best results were achieved by the LSI method. In contrast, among CBF recommenders, TFIDF performed considerably poorer, meaning that the sparsity of this representation hinders its ability to model the items (see Figure 4A). The optimal number of latent factors is 40, which must be assessed against the original vocabulary size (\(\sim\)14 K words) and the average transcript length (\(\sim\)3 K words). Other domains having larger vocabularies and textual representations can be expected to require more latent factors and vice versa. The KBF recommenders based on themes (COSIDE_IDF, DICE, and JACCARD) performed practically identically to TFIDF, meaning that the effort involved in representing the items using a controlled vocabulary of size \(\sim\)2 K did not provide much benefit. In spite of this finding, all the methods that used thematic representation in combination with the ontology managed to improve the results (with the exception of the JCN method). Therefore, the knowledge encoded in the ontology and the novel method for exploiting the ontology are useful in spite of the sparsity of the representation that was used. In addition, this result suggests that even a representation of relatively moderate dimensionality (\(\sim\)2 K entities) could benefit from a combination with a method such as LSI. Among the OBF recommenders, RES and LCH tied for the best performance. However, Figure 5 shows that RES is rather sensitive to the parameter \(p\), while LCH exhibited the best robustness on that matter. In addition, RES requires IC, which also requires the existence of a relatively large corpus to compute those statistics. The graphs in Figure 4 also show performance variation due to parameter \(k\), which is the number of nearest neighbors used in IKKN. The clear tendency is to get lower error rates as \(k\) increases. There is a general inflection point around \(k=40\) where more neighbors provide only a small benefit, and even an increased RMSE beyond this point for two of the best performing methods: LSI-40 and RES. Regarding the comparison of CBF/KBF/OBF recommenders against their CF counterparts, we observed that only the KBF recommenders fail to significantly outperform the CF ones in the Warm System scenario. It is important to note that, in that scenario, our modifications of the IKNN algorithm using LSI-40 and LCH do outperform the classical purely collaborative IKNN and remaining CF tested approaches. The most notable difference in the results of the Cold Start scenario is that all methods exhibit comparatively large variances across the 30 random partitions. Although all the IKKN variants manage to surpass all other CF recommenders, the latter do not produce significant statistical differences among them. Out of the IKNN approaches, the only statistically significant difference was observed between the CBF recommender (LSI-40) and the OBF recommenders (RES and LCH). Figure 5: Behavior of softness control parameter \(p\) from Equation (8) in the OBF recommenders when the number of nearest neighbors \(k\) is fixed at 60. The results also show that the CBF and OBF recommenders behave similarly in comparison to the other approaches, but the CBF recommender consistently outperforms the OBF one by a small margin. This difference is attributable to the particularly good quality of the content-based representation in the cause of our dataset. The fan curated Star Trek episode transcripts provide high-quality and detailed descriptions that even include fragments without dialog. This particular situation leads us to conclude that the effort invested in the construction of a detailed ontology is not worth it for recommendation purposes. However, that kind of representation is only possible for certain types of items (e.g., books, movies, TV shows) and, even if that is possible, its availability is not guaranteed.