Empowering Recommender Systems using Automatically Generated Knowledge Graphs and Reinforcement Learning (2025)

Abstract: Personalized recommender systems play a crucial role in direct marketing, particularly in financial services, where delivering relevant content can enhance customer engagement and promote informed decision-making. This study explores interpretable knowledge graph (KG)-based recommender systems by proposing two distinct approaches for personalized article recommendations within a multinational financial services firm. The first approach leverages Reinforcement Learning (RL) to traverse a KG constructed from both structured (tabular) and unstructured (textual) data, enabling interpretability through Path Directed Reasoning (PDR). The second approach employs the XGBoost algorithm, with post-hoc explainability techniques such as SHAP and ELI5 to enhance transparency. By integrating machine learning with automatically generated KGs, our methods not only improve recommendation accuracy but also provide interpretable insights, facilitating more informed decision-making in customer relationship management.

## introduction the increasing demand for personalized content has led to the development of recommendation systems that can effectively utilize structured information. knowledge graphs (kgs) have emerged as a promising solution for this challenge, offering improved recommendation performance and explainability due to the inherent comprehensibility of relationships between entities @xcite . a growing body of research is dedicated to exploring the potential of knowledge graph reasoning in personalized recommendation @xcite @xcite . one line of research focuses on knowledge graph embedding models, such as transe @xcite and node2vec @xcite , which align the knowledge graph in a regularized vector space, identifying the similarity between entities by calculating the distance between their representations @xcite . however, purely kg embedding-based approaches struggle to uncover multi-hop relational paths, limiting the ability to capture complex relationships between entities. another line of research investigates path-based recommendation techniques. gao et al. @xcite proposed the concept of metapaths for reasoning over kgs. although promising, this approach faces challenges when dealing with the numerous types of relations and entities present in large, real-world kgs, making it difficult to explore relationships between unconnected entities. wang et al. @xcite developed a path embedding approach for recommendation over kgs that enumerates all qualified paths between every user-item pair, followed by training a sequential rnn model to predict ranking scores for the pairs. while this method improves recommendation performance, it is not feasible to explore all paths for every user-item pair in large-scale kgs due to computational limitations. recent advances have focused on combining collaborative filtering (cf) with kg embedding techniques to enhance recommendation performance @xcite . for example, ai et al. @xcite proposed a method that incorporated a soft matching algorithm to identify explanation paths between users and items. however, this strategy generates explanations post-hoc through empirical similarity matching between user and item embeddings, providing retrospective rationales for the chosen recommendations rather than deriving explanations from the reasoning process @xcite . we argue that an intelligent recommendation agent should explicitly reason over knowledge graphs for decision-making rather than simply embedding the graph as latent vectors for similarity matching. in this paper, we treat knowledge graphs as a flexible structure to maintain the agent's knowledge about users, items, other entities, and their relationships. the agent initiates the process with a user and conducts explicit multi-step path reasoning over the graph, discovering suitable items for recommendation. this approach allows for the reasoning process to be easily interpreted, providing causal evidence for the recommended items. our goal is not only to select a set of candidate items for recommendation but also to provide the corresponding reasoning paths as interpretable evidence for each recommendation. to address the limitations of previous work, we propose an approach that casts the recommendation problem as a deterministic markov decision process (mdp) over the knowledge graph. we employ a reinforcement learning (rl) method, wherein an agent begins with a given user and learns to navigate to potential items of interest. the path in the kg then serves as an explanation for why the item should be recommended to the user. this approach presents three main challenges: measuring the correctness of an item for a user, efficiently exploring promising reasoning paths in the graph, and preserving the diversity of both items and paths during exploration. to tackle these challenges, we propose a kg-driven rl-based approach. the benefit of our approach is that it can also work when reviews or ratings of the items are not available and only click or other forms of response information is available to learn the user preferences. our experimental results demonstrate that our proposed method consistently outperforms stateof-the-art recommendation techniques, we present qualitative case studies to demonstrate the explainability of our approach, providing insights into the reasoning paths and decision-making processes of the recommendation agent. these case studies showcase the interpretability of our method, allowing users to better understand the rationale behind the recommendations. in summary, our research contributes to the growing body of literature on knowledge graph-based recommendation systems, specifically in the financial domain. by proposing a novel reinforcement learning approach and conducting a comparative study with the xg-boost algorithm, we offer valuable insights into the potential of knowledge graphs for improving the performance and explainability of personalized recommendation systems. our development of a kg-driven xgboost recommendation system further demonstrates the versatility and applicability of knowledge graph techniques in the field of recommendation. by developing a kg-driven xgboost recommendation system alongside our reinforcement learning approach, we aim to showcase the flexibility and potential of knowledge graph-based techniques in addressing a wide range of recommendation scenarios. our comparative study between the two approaches not only provides insights into their respective strengths and limitations but also highlights the importance of tailoring recommendation algorithms to specific application contexts and requirements. we have made public the source code of both the proposed approaches via a github link foot_0 . our main contributions are as follows: (1) automatic kg creation using structured and unstructured data. (2) use of kg for building an xgboostbased recommender system that can exploit click or other forms of response information. (3) use of kg for building an rl-based recommender system that can exploit click/response information. (4) explainability module that can explain the rationale behind the recommendations. the rest of the paper is structured as follows. in section 2, we describe the existing methods for building recommender systems. section 3 describes the methodology. section 4 describes the experimental setup. in section 5, we discuss and compare results in detail. finally, we conclude in section 6. ## methodology the problem addressed in this research is to provide a new type of recommendation, called knowledge graph driven explainable recommendation (kgdexr), that simultaneously performs item recommendation and path finding based on rich and heterogeneous information in the knowledge graph. the goal is to find a recommendation set of n items for a given user u from a subset of item entities i connected to user entities u through re-lations r ui in the knowledge graph g. the recommendation set should be associated with one reasoning path p j (u, i n ) (2 ≤ @xmath0, i n ) of user and recommended item, where j is the number of hops in the path and j is a given integer. the number of recommendations, n , is also given as an input. the knowledge graph g is defined as @xmath1, r, e t ), where e h is the head entity and e t is the tail entity in the kg. e h & e @xmath2, @xmath3, where e is the entity set and r is the relation set. a j-hop path from entity e 0 to entity e j is defined as a sequence of j + 1 entities connected by j relations, denoted by p the kgdexr problem can be formalized as finding a set of n items {i n } @xmath4, such that each pair (u, i n ) is associated with a reasoning path p j (u, i n ) (2 ≤ @xmath5. ## experimental setup in this section, we provide information on kg creation, kg embedding generation, and the data sets used in this work. ## results we have produced results using both kg-driven xgboost approach and kg-driven reinforcement learning approach. we constructed two kgs using unstructured data (educational article contents/texts) through saffron @xcite as mentioned in section 4. these kgs are "ukg_dp" and "ukg_cn" where u denotes unstructured data, dp denotes dependency parsing and cn denotes conceptnet. additionally, we created a kg referred to as "ckg" from both structured and unstructured data, as explained in section 4. the rationale behind using the ckg with rlbased approach is that it helps in generating explainable recommendations using paths in the ckg. for rl based approach we used kg embeddings generated using transe, as shown in we also compared the performance of our proposed approaches with state-of-the-art (sota) existing recommender systems. the existing recommender systems we used are: bpr (bayesian personalized ranking), neighborhood-based recommender system, ncf (neural collaborative filtering), and xgboost with sentence embedding. our kg-driven rl-based approach is explainable. to gain a better understanding of our model's interpretation of the recommendation, we present a case study based on the results obtained from our experiments. in this study, we analyze the path patterns uncovered by our model during the reasoning process, as well as examine different recommendation scenarios. as shown in figure 1 , the educational article highlighted with a blue dashed boundary is the article recommended by our rl-based model to a subscriber. we can see that the recommended article has some similarities with another educational article already recommended and clicked by that subscriber, therefore the model thinks that this article should be of relevance for that subscriber as the subscriber was interested in such kind of articles before. furthermore, our rl-based approach enables us to offer the top 10 educational articles for each subscriber. additionally, it can provide all the associated articles in the path that lead to the outcome, along with shared products/services, topics, and the most frequent common terms found in the text of the educational articles present in the path. our rl-based approach can provide such paths for each recommended item to a user which explain the results and play an important role in decision-making. to generate post-hoc explanation for kg driven xgboost-based approach, we used shap (lundberg and lee, 2017) and eli5 foot_3 . shap (shapley additive explanations) is a model-agnostic method used for explaining the output of machine learning models. it is based on game theoretic concepts and provides an explanation for each feature's contribution to the model's prediction. shap values quantify the impact of each feature by assigning a value to it, indicating how much it contributes to the prediction compared to the average prediction. shap relies on the concept of shapley values from cooperative game theory and it considers additive feature importance. figure 2 represents the kg-xgboost [ukg_cn] model's features with their average impact on the model output generated by shap. eli5 (explain like i'm 5) is a python library or framework for explainable machine learning models. eli5 focuses on understanding the overall behavior and importance of features in making predictions. eli5 reports feature importance using the "permutation importance" algorithm. figure 3 shows the kg-xgboost [ukg_cn] model's feature importance by assigning weights to the features based on their impact on the model output generated by eli5. both shap and eli5 show that click_frequency, kg_26, article_length, kg_32, kg_3, and kg_45 are the most important features that contributed most to the model results. overall, the proposed approaches are helpful in providing insights to understand the recommendations and simultaneously perform better than the existing baseline recommender systems. ## conclusion this research paper explores and demonstrates the use of knowledge graphs (kgs) to enhance personalized recommendations in the financial sector. we proposed two kg-driven recommender systems for a large multinational financial services company, utilizing reinforcement learning and the xgboost algorithm, respectively. the first approach employs reinforcement learning (rl), while the second utilizes the xgboost algorithm. the xgboostbased approach uses kg embeddings generated from both tucker and transe, and the rl-based approach uses transe-generated embeddings. we also performed experiments keeping the kg and the embedding same. the findings suggest that the kg-driven rl-based approach outperforms both the kg-driven xgboost system and baseline models, delivering more accurate and personalized educational article recommendations. additionally, the study emphasizes the importance of reasoning with knowledge for decision-making. overall, this study highlights the potential of combining advanced machine learning techniques with kg-driven insights to improve customer experience and drive business growth in the investment sector.