P er KGQA : Question Answering over Personalized Knowledge Graphs (2022)

Abstract: Previous studies on question answering over knowledge graphs have typically operated over a single knowledge graph (KG). This KG is assumed to be known a priori and is lever- aged similarly for all users’ queries during inference. However, such an assumption is not applicable to real-world settings, such as health- care, where one needs to handle queries of new users over unseen KGs during inference. Furthermore, privacy concerns and high computational costs render it infeasible to query the single KG that has information about all users while answering a specific user’s query. The above concerns motivate our question answer- ing setting over personalized knowledge graphs (PERKGQA) where each user has restricted access to their KG. We observe that current state-of-the-art KGQA methods that require learning prior node representations fare poorly. We propose two complementary approaches, PATHCBR and PATHRGCN for PERKGQA. The former is a simple non-parametric technique that employs case-based reasoning, while the latter is a parametric approach using graph neural networks. Our proposed methods circumvent learning prior representations, can generalize to unseen KGs, and outperform strong baselines on an academic and an internal dataset by 6.5% and 10.5%.

## introduction the task of question answering over knowledge graphs (kgqa), involves answering a natural language question by querying a predefined knowledge graph (kg), such as wikidata or freebase. progress in kgqa research has addressed several challenges, such as answering complex questions, multi-hop reasoning, @xcite , conversational kgqa @xcite , and multi-lingual kgqa @xcite , and has also found applications in tax, in- * work done during internship at amazon aws ai labs. most kgqa research has focused on generalizable or generic knowledge, which assumes there is a predefined global kg for all queries. this assumes that nodes used during inference were already defined in the kg during training and holds for cases that focus on generalizable knowledge. this work proposes approaches that circumvent the need to make such an assumption. furthermore, using a single global kg to handle queries of different users raises additional concerns, especially when a user's query requires situated knowledge such as personal information. in this paper, we formulate perkgqa or question answering over personalized knowledge graphs. here the user has access to their specific kg, a subset of the global kg that contains only the information relevant to the user. we are restricted to the user's kg to answer their queries during training and inference. such a setting addresses the challenges above of scalability, privacy, and generalizing over unseen kgs. perkgqa appears deceivingly simple in conception since we afford access to a subset of the larger global kg. one can claim that our setting is similar to the kgqa subtask where subgraphs and questions are predefined, and thus, traditional kgqa methods are applicable. however, information retrieval based kgqa methods employ knowledge graph completion techniques like transe @xcite to learn node representations over the global kg and reuse them during inference. alternately, other approaches leverage additional information such as semantic parses, logical forms, and query graphs to answer queries. this sets perkgqa apart because we lack access to any prior information, be it text, semantic parses, or prior representations of kg nodes. our setting requires learning node representations from scratch for each kg to handle unknown entities during inference. moreover, other challenges prevalent in kgqa settings, namely multi-hop reasoning or answering complex/constraint-based questions, are also applicable to perkgqa. to the best of our knowledge, we are the first to address the challenges of kgqa over unseen kgs in the absence of any additional information. we propose two approaches, pathcbr and pathrgcn, that are well-suited to these settings. pathcbr is a simple non-parametric case-based reasoning approach that encodes path information of past queries to answer a new query. pathrgcn is a parametric approach that employs graph neural networks, path information, and the kg's structure to extract answers. these approaches circumvent the need for learning prior node representations and can be readily applied to unseen kgs. ## datasets we operate on two datasets: an internal dataset, cloudkgqa, built on top of cloudkg, and an academic dataset called mod-webqsp designed to mimic our setting. an instance in either dataset follows the same task formulation in section 2, namely, for each question q, there exists a corresponding kg, k q , which contains all the necessary information. also, each question q is associated with one or more source entities; these correspond to nodes in the k q linked through salient mentions of entities in q. e.g., the source entity for, "who was responsible for lincoln's assassination?" is the node corresponding to abraham lincoln. user 101 res_2 res_3 res_1 resource_id resource_id resource_id has tags has_tags chatbot service tts module india has_service has_service has_service has_service has_region abc123 has_tags nlp-serv demo_1 has_region us has_region user 102 res_4 res_5 eu def456 cv_res cv analytics resource_id resource_id has_tags has_tags has_region has_region has_service has_service which resources have nlp-serv and demo_1 tags? where is my res_4 resource? kg 1 kg 2 q 2 q 1 perkgqa system res_1, res_3 eu user 101 user 102 ## results in this section, we pose the following research questions (rqs) and attempt to answer the same. we present instances of preprocessed questions that serve as input to the model. rq1. how do our proposed approaches fare on perkgqa compared to kgqa baselines? we observe that both pathcbr and pathrgcn, yield the highest performance on cloudkgqa, outperforming the existing baselines by over 100% for hits@1 and accuracy in for mod-webqsp, pathrgcn achieves the highest performance outperforming preexisting baselines significantly (p-@xmath0.001). however, pathcbr achieves performance comparable to the baselines, and can answer questions corresponding to templates encountered during training, for instance, "who plays ken barlow in coronation". we attribute the low performance of pathcbr to: thus if the templates are widely different, it is not sufficient to encode the question using a ptlm; rather, we need to fine-tune the questions to learn meaningful representation. we further inspect the capabilities of our techniques to address the individual characteristics of perkgqa, namely multiple answers, variable hop distance, multiple constraints, and variable kg size. our approaches outperform baselines consistently and significantly on all such fronts. a thorough analysis of our proposed approaches to the different properties of these two datasets reveals their complementary strengths. we note pathrgcn has a better performance on larger kg size, more answers, longer hops, and additional constraints, and vice-versa for pathcbr. we defer the reader to appendix c for these results. ## conclusion and future work we propose perkgqa, a realistic setting for performing question answering over knowledge graphs; for each user's question, we have their corresponding kg but no additional information. such a setting addresses the challenges of unseen nodes during inference, and prevents access to information of other users while being computationally feasible. however, state-of-the-art kgqa techniques that require learning node representations a priori fare poorly. we propose two approaches, a simple non-parametric case-based-reasoning model and a supervised neural architecture, harness path information for qa. our approaches improve upon the baselines by 6.5% on an academic dataset and 10.5% on an internal dataset. having demonstrated the applicability of perkgqa in the cloud service provider domain, we aim to explore other scenarios involving personalized or sensitive information, like healthcare. prior work in medical nlp has focused predominately on generic or ontological knowledge such as umls. a personalized kg, constructed over a patient's health records, will encode information specifically for the individual and not the general population, e.g. whether the patient is allergic to certain medications. we plan to collaborate with medical professionals and create personalized kg in the healthcare domain to assist patients. furthermore, we seek to address certain limitations of our current approach, namely their inability to tackle spurious paths. we plan to rectify it either by explicitly providing the correct path information or incorporating some learning paradigm to detect them @xcite . moreover, for pathcbr, the retrieval phase is a bottleneck since one needs to compare a given query with all possible training questions and requires better indexing schemes like faiss @xcite . likewise, the inference time for both approaches increases as the number and the length of the paths increase. however, pathrgcn can adapt to longer paths since the node embeddings provide some degree of additional supervision. nevertheless, we plan to explore techniques beyond embedding-based approaches, namely semantic parsing or query-graph generation, to alleviate the path-based constraint and adapt them to our perkgqa. in the absence of gold logical forms, we plan to learn semantic parses through a weakly-supervised or distantlysupervised setting similar to @xcite . the cloudkgqa dataset, where certain entities were universal, like names of regions (india, usa). this would facilitate prior kgqa techniques, like embedkgqa, that perform kgc on the individual kgs to share embeddings and perform better. however, our proposed approaches, pathcbr, and pathrgcn remain agnostic to the degree of overlap. they do not keep track of any prior entities. specifically pathcbr masks these entities in the question, whereas pathrgcn learns these entity representations from scratch for each kg.