Graph Reasoning for Question Answering with Triplet Retrieval (2023)

Abstract: Answering complex questions often requires reasoning over knowledge graphs (KGs). State-of-the-art methods often utilize entities in questions to retrieve local subgraphs, which are then fed into KG encoder, e.g. graph neural networks (GNNs), to model their local structures and integrated into language models for question answering. However, this paradigm constrains retrieved knowledge in local subgraphs and discards more diverse triplets buried in KGs that are disconnected but useful for question answering. In this paper, we propose a simple yet effective method to first retrieve the most relevant triplets from KGs and then rerank them, which are then concatenated with questions to be fed into language models. Extensive results on both CommonsenseQA and OpenbookQA datasets show that our method can outperform state-of-the-art up to 4.6% absolute accuracy.

## introduction answering complex questions is a challenging task since it often requires world knowledge and reasoning capability of underlying models @xcite @xcite . pretrained language models, e.g. bert @xcite and roberta @xcite , have shown promising results by fine-tuning on downstream question answering tasks. however, world knowledge and reasoning of these models are learned from unstructured data, e.g. wikipedia text, and are still limited @xcite . on the other hand, there exist large-scale knowledge graphs (kgs), e.g. freebase @xcite and conceptnet @xcite , capturing world knowledge explicitly by triples to record relations between entities @xcite . however, how to effectively integrate kgs into language models for question answering is still * work was done during internship at amazon. an open research problem. @xcite al. (2019); yu et al. (2020); ye et al. (2019); zhang et al. (2019) ; @xcite focus on utilizing kgs to construct distant supervision signals for continuous pre-training, however, kgs are often dynamic in practice and it is often hard to edit knowledge in models without further training, limiting their usage. @xcite ; @xcite linearize reasoning paths in kgs and train language models on them to generate novel knowledge triplet during inference. however, kgs are discarded after training and language models can hallucinate false world knowledge @xcite . 2022 ) instead first recognize entities in questions and link them to kgs to retrieve subgraphs as additional input besides questions. however, this paradigm constrains retrieved knowledge in local subgraphs and discards more diverse triplets buried in kgs that are disconnected but useful for question answering. in addition, they require extra kg encoders with parameters trained from scratch besides standard language models, limiting model performance when training data is limited. recently, there have been growing interests to convert kg as a list of passages represented as natural languages. oguz et al. (2022 retrieval following oguz et al. (2022); ma et al. (2021a); li et al. (2022); zha et al. (2021). the overall pipeline of proposed method is shown in figure 1 . specifically, we first linearize kg into triplets and convert them into passages by templates and directly retrieve the most relevant ones by questions with both sparse bm25 @xcite and dense passage retriever (dpr) @xcite . we then rerank these passages by pre-trained cross-encoders @xcite . finally, most relevant passages and questions are linearly concatenated and fed into pre-trained language model for question answering. this paradigm has several advantages compared to recent state-of-the-art @xcite @xcite : (1) it is simple yet effective and can outperform state-of-the-art complicated question answering systems up to 4.6% absolute accuracy (2) it does not need extra kg encoders trained from scratch, e.g. gnns @xcite , and simply fuses knowledge passages and questions for question answering by standard language models. ## conclusion in this paper, we propose a simple but effective method for question answering over knowledge graphs with triplet retrieval. extensive experiments on two datasets show that our method can consistently outperform state-of-the-art. ablation study further shows that our model benefits from both reranking module and the combination of sparse and dense retrievers. we believe that our work can inspire future research for question answering over knowledge graphs.