Conversational Question Answering with Language Models Generated Reformulations over Knowledge Graph (2024)

Abstract: Conversational question answering (ConvQA) over knowledge graphs (KGs) involves answering multi-turn natural language questions about information contained in a KG. State-of-the-art methods of ConvQA often struggle with inexplicit question-answer pairs. These inputs are easy for human beings to understand given a conversation history, but hard for a machine to interpret, which can degrade ConvQA performance. To address this problem, we propose a reinforcement learning (RL) based model, CoRnNet, which utilizes question reformulations generated by large language models (LLMs) to improve ConvQA performance. CoRnNet adopts a teacher-student architecture where a teacher model learns question representations using human writing reformulations, and a student model to mimic the teacher model’s output via reformulations generated by LLMs. The learned question representation is then used by a RL model to locate the correct answer in a KG. Extensive experimental results show that CoRnNet outperforms state-of-the-art ConvQA models.

## introduction knowledge graphs (kgs) are collections of nouns represented as nodes (representing real-world entities, events, and objects) and edges (denoting relationships between nodes). knowledge graph question answering (kgqa) has long been a focus of study, with the goal of answering queries using information from a kg. however, traditional kgqa approaches often only consider single-shot questions @xcite , rather than the iterative nature of real-world conversation. conversational question answering (convqa) addresses this gap by allowing users to interact with a qa system conversationally. convqa systems have had much success, as seen by google's lambda @xcite , amazon's alexa and openai's chat-gpt. conversational question answering (convqa) involves a multi-turn process consisting of users iteratively asking natural language questions, a system deciphering both the conversation context and underlying queries, and the system returning natural language answers. some models will create rich, human-like responses @xcite @xcite , these methods are known as 'dialogue' conversation models. while for convqa over kgs, a corresponding entity in the kg is sufficient to answer the input question, we call it 'non-dialogue' conversation models. in this paper, we focus on the non-dialogue con-vqa task as shown in example 1. example 1: q 1 : who is the author that wrote the book moby-dick? reformulation1: author of the book? reformulation2: who wrote moby dick? a 1 : herman melville q 2 : when was he born? reformulation1: his birthdate is? reformulation2: when was herman melville born? a 2 : 1 august 1819 q 3 : and where is he from originally? reformulation1: his place of birth? reformulation2: where did he grow up? a 3 : manhattan q 4 :how about his wife? reformulation1: where is herman melville's wife from? reformulation2: herman melville's wife's place of birth? a 4 : boston q 5 : did they make a movie based on the book? a 5 : yes in general, a conversation is typically initiated with a well-formed question (i.e., q 1 ) followed by inexplicit follow-up questions (e.g., q 2q 5 ). the initial question (q 1 ) often includes a central topic entity of interest ("moby-dick"), while the topic entities of follow-up questions (q 2q 5 ) are not explicitly given. additionally, the topic entity of the conversation may shift over time (e.g., inquiring about the birth time of herman melville in q 2 ). to operate convqa over kg, different methods have previously been proposed. for instance, @xcite use named entity recognition (ner) methods to detect potential kg topic entities in the conversation and employ multi-agent reinforcement learning starting from these entities to find answers; the performance of this method is largely dependent on the quality of the detected entities. @xcite propose finding a conversationrelated subgraph and using heuristic-based methods to identify the answer within the subgraph. the subgraph is expanded as new questions are asked. @xcite use contrastive learning to separate correct answers from incorrect answers. despite the above progress, inexplicit input data hinders a convqa system's ability to find correct answers @xcite @xcite . two common linguistic phenomena which undermine the semantic completeness of a query in the conversation are: anaphora and ellipsis @xcite . anaphora refers to the phenomenon of an expression that depends on an expression in the previous context. in example 1, the word "he" in q 2 refers to a foot_0 . meanwhile, ellipsis refers to the phenomenon of the omission of expressions in the previous context. for example, the complete form q 4 should be "where is herman melville's wife from?". to address this issue, several methods aim to learn a reformulation of the input query, rewriting the original question in a more meaningful way. then, one can search for the answer using this new reformulation with existing techniques @xcite @xcite . although many of the existing question rewriting models have shown potential to enhance convqa performance, as demonstrated by prior research @xcite , their generated reformulations fall short compared to human-generated reformulations @xcite . in this paper, we present cornnet, a new reinforcement learning (rl) model for non-dialogue conversational question answering (convqa) with large language model (llm) generated reformulations. first, we fine-tune existing llms, gpt2 @xcite and @xcite , to generate high quality reformulations, using human writing reformulations as the ground truth. second, to further increase the convqa performance, we propose a teacher-student architecture to achieve near human-level performance. 1 specifically, cornnet (1) directly trains a teacher model with human writing reformulations in the training data, and (2) indirectly trains a student model with llm-generated reformations to mimic the teacher model's output so that it can approach human-level performance. note that the human writing reformulations only exist in the training and validation data. lastly, to locate an answer, a rl model walks over the kg, sampling actions from a policy network to guide the direction of the walk and identify candidate answers. our experiments demonstrate the effectiveness of cornnet and its superiority over the state-of-the-art conversational question answering baselines. the main contributions of this paper are: (1) analysis. we demonstrate that although llms are good question reformulators, their performance lags behind human-level performance. ( 2 ) method. we propose a rl based model cornnet which utilizes the question reformulations to improve the qa performance. the proposed teacher-student model can help us achieve near human-level performance with llm-generated reformulations. (3) evaluation. the experimental results on several real-world datasets demonstrate that the proposed cornnet consistently achieves state-of-the-art performance. ## problem definition a kg can be denoted as @xmath0, r, l) where @xmath1, v 2 , ..., v n } is the set of nodes/entities, @xmath2, r 2 , ..., r m } is the set of relations and l is the list of triples. each triple in the kg can be denoted as (h, r, t) where @xmath3.e., subject) of the triple, @xmath4.e., object) of the triple and @xmath5.e., relation, predicate) of the triple which connects the head h to the tail t. the embedding of a node or relation type is represented by bold lowercase letters, e.g., e i , r i . each triple/edge (h, r, t) in the kg has a unique edge embedding which is denoted as u r . conversational question answering over a kg aims to iteratively answer multiple related questions from the users. unlike dialog question answering which wants the chatbot to imitate the response of a human, convqa over kg only requires the model to return entities in the knowledge graph. we formally define the key terminologies used in this paper as follows. ## proposed method due to anaphora and ellipsis, current convqa methods often rewrite input queries to generate more understandable reformulations. in this paper, we follow this general idea by fine-tuning two existing llms, gpt2 and bart, to generate reformulations. although gpt2 and bart are good reformulation generators, their performance still lags behind human-level performance. to further improve the performance, we propose a teacherstudent architecture. the teacher model learns the question representation by using human written reformulations, while the student model takes reformulations generated by llms as input, and tries to mimic the output of the teacher model, so that it can achieve the same performance as the teacher model despite using the llm-generated reformulations. in each iteration, our model uses the conversation history and the current query to identify the current topic entity, and a rl agent travels the kg starting from the topic entity to find the answer. this process is repeated for a number of turns until the conversation is completed. figure 1 illustrates the framework of the proposed cornnet. we will describe the details of each component in the following subsections. ## experiments we use two datasets in the experiments: con-vquestions @xcite and @xcite . convquestions contains a total of 6,720 conversations, each with 5 turns. con-vref contains a total of 6,720 conversations, each with 5 turns. the code and datasets will be made publically available upon the acceptance of the paper. the details of these datasets can be found in appendix. both convquestions and convref use wikidata foot_1 as their background kg. however, the full wikidata kg is extremely large, containing approximately 2 billion triples. therefore, in the experiment, we sample a subset of triples from wikidata. we first take the overlapped entities between the wikidata and the qa datasets, and then we further obtain all the one-hop neighbours of these overlapped entities. the one-hop neighbors are retrieved from both the original data dump and also the entities' corresponding online wikidata websites. we compare the performance of our method, cornnet, with four baselines: convex (christmann et al., 2019), conqer @xcite , @xcite . the details of all baselines can be found in appendix. two llms are used to generate reformulations for the input query, which are gpt2 @xcite and @xcite . for each input question, we generate multiple reformulations and use attention mechanism to aggregate them inside the model. we adopt the following ranking metrics which are also employed by the previous baselines: (1) precision at the top rank (p@1); (2) mean reciprocal rank (mrr); (3) hit ratio at k (h@k/hit@k). the details of all the datasets and experiment environment can be found in appendix. ## ethical considerations we have thoroughly evaluated potential risks associated with our work and do not anticipate any significant issues. our conversational question answering framework is intentionally designed to prioritize usability and ease of implementation, thereby reducing barriers for adoption and minimizing operational complexities. furthermore, it's essential to highlight that our research builds upon an opensource dataset. this ensures transparency, fosters collaboration, and helps address ethical considerations by providing accessibility to the underlying data. ## limitation our dataset exhibits several limitations that warrant consideration. firstly, our training data is constrained by its limited scope, primarily focusing on specific domains rather than providing comprehensive coverage across diverse topics. this restriction may affect the model's generalizability and performance in addressing queries outside of these predefined domains. moreover, while our knowledge graph serves as a valuable resource for contextual information, it is essential to acknowledge its incompleteness. despite its vast size, certain areas within the knowledge graph may lack sufficient data or connections, potentially leading to gaps in the model's understanding and inference capabilities. ## acknowledgements ll, bh and ht are partially supported by nsf @xcite , darpa (hr001121c0165), and nifa (2020-67021-32799).