Dynamic Heterogeneous-Graph Reasoning with Language Models and Knowledge Representation Learning for Commonsense Question Answering (2023)

Abstract: Recently, knowledge graphs (KGs) have won noteworthy success in commonsense question answering. Existing methods retrieve relevant subgraphs in the KGs through key entities and reason about the answer with language models (LMs) and graph neural networks. However, they ignore (i) optimizing the knowledge representation and structure of subgraphs and (ii) deeply fusing heterogeneous QA context with subgraphs. In this paper, we propose a dynamic heterogeneous-graph reasoning method with LMs and knowledge representation learning (DHLK), which constructs a heterogeneous knowledge graph (HKG) based on multiple knowledge sources and optimizes the structure and knowledge representation of the HKG using a two-stage pruning strategy and knowledge representation learning (KRL). It then performs joint reasoning by LMs and Relation Mask Self-Attention (RMSA). Specifically, DHLK filters key entities based on the dictionary vocabulary to achieve the first-stage pruning while incorporating the paraphrases in the dictionary into the subgraph to construct the HKG. Then, DHLK encodes and fuses the QA context and HKG using LM, and dynamically removes irrelevant KG entities based on the attention weights of LM for the second-stage pruning. Finally, DHLK introduces KRL to optimize the knowledge representation and perform answer reasoning on the HKG by RMSA.We evaluate DHLK at CommonsenseQA and OpenBookQA, and show its improvement on existing LM and LM+KG methods.

## introduction question answering (qa) is a challenging task that requires machines to understand questions asked by natural language and respond to the questions based on the knowledge acquired. recently, qa has made remarkable progress with the development of language models (lms) @xcite @xcite . fine-tuning based on lms has now become a major paradigm for qa tasks. lms are pre-trained on a general large-scale corpus containing rich world knowledge, which the machine can utilize when fine-tuning downstream tasks using lms. in some simple, fact-based qa tasks, such as squad @xcite and race @xcite , machine has surpassed humans in terms of answer accuracy. however, the machine remains less satisfactory in some structured reasoning qa tasks that require commonsense knowledge. commonsense knowledge is the general law summarized by human beings through observation, research, and reflection of various phenomena in the objective world, which is verified by the longterm experience of countless people and is the common daily consensus of people. when humans answer questions, they use this knowledge unconsciously. for example, if you ask "john had an urgent matter to attend to at his company, and he drove fast to the company but stopped at an intersection, what could have happened?". we can reason that john may be passing through the intersection when the traffic light turns red. thus, he has to stop and wait for the light to turn green. this commonsense reasoning is easy for humans. however, considering that commonsense knowledge is a relatively tacit knowledge, lms do not capture it well. knowledge graphs (kgs) store a large amount of commonsense knowledge that can be used by machines to make sound judgments, and this knowledge can provide the machines with displayed and interpretable evidence. therefore, some methods @xcite @xcite @xcite have introduced kgs into lms-based qa methods to model and reason about structured knowledge in kgs through graph what furniture will you normally find near a side chair? a. bedroom b. table* c.wheel barrow d. building e. office qa context knowledge paths and paraphrases relatedto relatedto side bank table relatedto usedfor side top table madeof relatedto furniture wood table isa relatedto furniture desk table relatedto chair ¾¾¾¾¾ ® ¾¾¾¾¾ ® ¾¾¾¾¾ ® ¾¾¾¾¾ ® ¾¾¾¾¾ ® ¾¾¾¾¾ ® ¾¾¾ ® ¾¾¾¾¾ ® isa desk table atlocation atlocation chair cat table atlocation atlocation side chair room table atlocation atlocation side chair bedroom table ...... side chair : ¾¾¾¾¾ ® ¾¾¾ ® ¾¾¾¾¾® ¾¾¾¾¾® ¾¾¾¾¾® ¾¾¾¾¾® ¾¾¾¾¾® ¾¾¾¾¾® a straight-backed chair without arms. a piece of furniture with tableware for a meal laid out on it. ...... figure 1 : an example from commonsenseqa , we retrieve knowledge paths from conceptnet @xcite and key entity paraphrases from wordnet @xcite and wiktionary. currently, combining lms and gnns to solve commonsense qa (cqa) task has proven to be an effective method but still contains some problems: (i) in the key entities-based subgraph extraction method, the goodness of the key entities largely determines the quality of the subgraph. as shown in figure 1 , entities such as "wood", "bank", "top", and "cat" are some noisy knowledge for the current question, and they affect the model's judgment during the inference process. but a part of the noisy knowledge can be solved by optimizing key entities. in the example of figure 1 , "side chair" is a noun phrase, which should be considered as a whole when retrieving knowledge based on it, and this will reduce the introduction of some noisy knowledge; (ii) the knowledge representation of entities in subgraph are mostly obtained by glove @xcite , lms, and so on, ignoring the semantic associations between entities; additionally, the knowledge representations obtained are less effective; (iii) given that the qa context and subgraph have different structures, existing methods encode qa context and subgraph separately, with shallow interactions only at the gnn layer through message passing @xcite or at the output layer through attention mechanism @xcite or mlp @xcite @xcite , lacking deep fusion of qa context and subgraph, which will hinder the inference capability of the model. based on the above problems, we propose a dynamic heterogeneous-graph reasoning method based on language models and knowledge representation learning (dhlk). specifically, given a question and choice, we first use keybert (grootendorst, 2020) to extract the candidate entities and introduce wordnet @xcite and wiktionary foot_0 vocabularies to filter the candidate entities and then obtain the key entities, which can remove some noisy entities in the subgraph retrieval process and realize first-stage pruning of the subgraph. we also incorporate the paraphrases of key entities in the two dictionaries as entities into the subgraph to construct a heterogeneous knowledge graph (hkg). then, we use lm to encode the qa context and hkg and fuse the qa context and hkg in the encoding process. in addition, we dynamically remove irrelevant entities according to the attention weights of lm to achieve the second-stage pruning of the subgraph. finally, we combine krl and relation mask self-attention (rmsa) to optimize the knowledge representation of hkg and incorporate the path information in the hkg into the qa context. in summary, our contributions are threefold: ## related work recently, large lms such as unifiedqa @xcite , t5 @xcite and gpt-3 @xcite have been widely applied in qa tasks, such as open-domain question answering (odqa) and cqa, driving the development of qa. however, larger lms result in disproportionate resource consumption and training time. therefore, many works have enhanced the reasoning ability of machines by introducing external knowledge, hoping to achieve good answering results while reducing resource consumption and training time. knowledge-enhanced odqa. odqa model utilizes external knowledge to answer questions, typically consisting of a retriever and a reader component. with the development of lms, retrieval augmented architectures @xcite have become the mainstream method for odqa. they apply lms to retriever-reader and conduct joint training of the retriever-reader. however, previous works @xcite primarily focused on unstructured knowledge sources, such as wikipedia. recently, some works @xcite @xcite have started incorporating structured kgs into the retriever-reader architecture to enhance retrieval effectiveness and question answering capabilities. for example, unik-qa @xcite converts kg triplets into text and merges them with unstructured knowledge repositories. kg-fid @xcite utilizes kg to establish relational dependencies between retrieved paragraphs and employs gnns to sort and prune the retrieved paragraphs. grape @xcite constructs a localized bipartite graph for each pair of question and article, learning knowledge representations through gnns. knowledge-enhanced cqa. cqa also requires external knowledge to answer questions, but it is more focused on commonsense questions. from the perspective of knowledge and qa context fusion, there are currently two main methods. some works @xcite @xcite feed the retrieved knowledge together with the qa context into the lm, utilizing self-attention to fuse the knowledge. however, the self-attention treats the input knowledge and qa context indiscriminately, which can undermine the semantic information of the qa context. other works @xcite @xcite @xcite combine lm and gnns to solve cqa. for example, qagnn @xcite uses lm to estimate the importance of subgraph entities and considers the qa context as an additional node connected to the subgraph. jointlk @xcite uses the bidirectional attention module to fuse the two modalities while designing a pruning module to remove irrelevant entities from the subgraph. greaselm @xcite fuses encoding representations from lm and gnns through multi-layered modality interaction operations. however, these works encode the qa context and kg subgraph in isolation, leading to limited interaction between textual and kg representations. additionally, they don't consider the influence of key entities and knowledge representations on subgraph retrieval and model inference. in contrast to previous works, we propose to reduce noisy knowledge by optimizing the set of key entities in the subgraph retrieval process. in addition, we use lm to encode and fuse the two modalities and prune the subgraph according to the attention weights of lm. meanwhile, during the inference process, we introduce the krl algorithm to optimize the knowledge representation of the subgraph. figure 2 shows the overall architecture of our method. ## conclusions in this paper, we propose dhlk, a cqa method based on lm and krl. our main innovations include: (i) constructing the hkg based on kg and dictionary, and introducing a two-stage pruning strategy and krl to optimize the structure and knowledge representation of the hkg; (ii) deeply fusing the qa context and hkg in the encoding stage of lm, and designing a kg2qa module to incorporate the paths information of hkg into the qa context. the effectiveness of dhlk is demonstrated via experimental results and analysis on commonsenseqa and openbookqa.