Dynamic Relevance Graph Network for Knowledge-Aware Question Answering (2022)

Abstract: This work investigates the challenge of learning and reasoning for Commonsense Question Answering given an external source of knowledge in the form of a knowledge graph (KG). We propose a novel graph neural network architecture, called Dynamic Relevance Graph Network (DRGN). DRGN operates on a given KG subgraph based on the question and answers entities and uses the relevance scores between the nodes to establish new edges dynamically for learning node representations in the graph network. This explicit usage of relevance as graph edges has the following advantages, a) the model can exploit the existing relationships, re-scale the node weights, and influence the way the neighborhood nodes’ representations are aggregated in the KG subgraph, b) It potentially recovers the missing edges in KG that are needed for reasoning. Moreover, as a byproduct, our model improves handling the negative questions due to considering the relevance between the question node and the graph entities. Our proposed approach shows competitive performance on two QA benchmarks, CommonsenseQA and OpenbookQA, compared to the state-of-the-art published results.

## introduction solving question answering (qa) problems usually requires both language understanding and human commonsense knowledge. large-scale pretrained language models (lms) have achieved success in many qa benchmarks @xcite @xcite . however, lms have difficulties in predicting the answer when reasoning over external knowledge is required @xcite . therefore, using the external sources of knowledge explicitly in the form of knowledge graphs (kgs) is a recent trend in question answering models @xcite , and wrong answer entity nodes (orange boxes), we predict the answer by reasoning over the question and the extracted kg subgraph. mark, shows an example for which answering the question requires commonsense reasoning. in this example, the external kg provides the required background information to obtain the reasoning chain from question to answer. we highlight two issues in the previous approaches taken to solve this qa problem: a) the extracted kg subgraph sometimes misses some edges between entities, which breaks the chain of reasoning that the current models can not exploit the connections, b) the semantic context of the question and connection to the answer is not used properly, for example, reasoning when negative terms exist in the question, such as no and not, is problematic. the first above-mentioned issue is caused by the following reasons. first, the knowledge graph is originally imperfect and does not include the required edges. second, when constructing the subgraph, to reduce the graph size, most of the models select the entities that appear in two-hop paths @xcite . therefore, some intermediate concept (entity) nodes and edges are missed in the extracted kg subgraph. in such cases, the subgraph does not contain a complete chain of reasoning. third, the current models often cannot reason over paths when there is no direct connection between the involved concepts. while finding the chain of reasoning in qa is challenging in general @xcite , here this problem is more critical when the kg is the only source and there are missing edges. looking back at figure 1 , the kg subgraph misses the direct connection between guitar and playing instrument (green arrow). for the issue of considering question semantics, as @xcite points out, previous models are not sensitive to the negation words and consequently predict opposite answers. qa-gnn @xcite model is the first work to deal with the negative questions. qa-gnn improves the reasoning under negation, to some extent, by adding the qa global node to the graph. however, the challenge still exists. to solve the above challenges, we propose a novel architecture, called dynamic relevance graph network (drgn). the motivation of our proposed drgn is to recover the missing edges and establish direct connections between concepts to facilitate multi-hop reasoning. in particular, drgn model uses a relational graph network module while influencing the importance of the neighbor nodes using an additional relevance matrix. it potentially can recover the missing edges to establish a direct connection based on the relevancy of the node representations in the kg during the training. the module can potentially capture the connections between distant nodes while benefiting from the existing kg edges. our proposed model learns representations directly based on the relevance scores between subgraph entity pairs that are computed by inner product operation. at each convolutional layer of the graph neural network, we compute the inner product of the nodes based on their current layer's node representations dynamically and build the neighborhoods based on this relevance measure and form a relevance matrix accordingly. this can be seen as a way to learn new edges as the training goes forward in each layer while influencing on the weights of the neighbors dynamically based on their relevance. as shown in figure 1 , the relevance score between guitar and playing instrument is stronger than other nodes in the subgraph. moreover, since the graph includes the question node, the relevance between the question node and entity nodes is computed at every layer, making use of the contextual information more effectively. it becomes more clear that the student will spend the free period in the music room rather than the concert. in summary, the contributions of this work are as follows: 1) the proposed drgn architecture exploits the existing edges in the kg subgraph while explicitly uses the relevance between the nodes to establish direct connections and recover the possibly missing edges dynamically. this technique helps in capturing the reasoning path in the kg for answering the question. 2) our model exploits the relevance between question and the graph entities, which helps considering the semantics of the question explicitly in the graph reasoning and boosting the performance. in particular, it improves dealing with the negation. 3) our proposed model obtains competitive results on both commonsenseqa and openbookqa benchmarks. our analysis demonstrates the significance and effectiveness of the drgn model. furthermore, the textual context is used as an additional node in the kg subgraph. for example, @xcite and @xcite introduce the sentence node into the graph, while @xcite and @xcite add the paragraph node and question+ cand_ans att q k v mul add & norm add & norm feed forward context encoder kg dynamic relevance matrix 1 dynamic relevance matrix 2 language rep question node rep kg subraph pooling rep mlp answer prediction language rep kg subgraph kg subgraph adding question information figure 2: our proposed drgn model is composed of the language context encoder module, kg subgraph construction module, graph neural network module, and answer prediction module. the blue color entity nodes represent the entity mentioned in the question. the yellow color node represents the answer node. the red color node is the question node. we use different colors to draw the dynamic relevance matrix 1 and 2 because the relevance matrix changes dynamically in each graph neural layer. ## conclusion in this paper, we propose a novel dynamic relevance graph network (drgn) architecture for commonsense question answering given an external source of knowledge in the form of a knowledge graph. our model learns the graph node representation while a) exploits the existing relations in kg, b) re-scales the importance of the neighbor nodes in the graph based on training a dynamic relevance matrix, c) establishes direct connections between graph nodes based on measuring the relevance scores of the nodes dynamically during training. the dynamic relevance edges help in finding the chain of reasoning when there are missing edges in the original kg. our quantitative and qualitative analysis shows that the proposed approach facilitates answering the complex questions that need multiple hops of reasoning. furthermore, since drgn uses the relevance between the question node and graph entities, it exploits the richer semantic context of the question in graph reasoning which leads to improvements in the performance on the negative questions. our proposed approach shows competitive performance on two qa benchmarks, including commonsenseqa and openbookqa.