Learning Contextualized Knowledge Graph Structures for Commonsense Reasoning

1 INTRODUCTION . Commonsense knowledge is essential for developing human-level artificial intelligence systems that can understand and interact with the real world . However , commonsense knowledge is assumed by humans and thus rarely written down in text corpora for machines to learn from and make inferences with . Fig . 1 shows an example in a popular commonsense reasoning benchmark named CommonsenseQA ( Talmor et al. , 2019 ) . The knowledge about the relations between concepts , e.g. , the fact triple ( print , Requires , use paper ) , is not explicitly given in the question and answer . Without important background knowledge as clues , natural language understanding ( NLU ) models may fail to answer such simple commonsense questions that are trivial to humans . Current commonsense reasoning models can be classified into retrieval-augmented methods ( Banerjee et al. , 2019 ; Pan et al. , 2019 ) and KG-augmented methods ( Wang et al. , 2019b ; Kapanipathi et al. , 2020 ) . Retrieval-augmented methods retrieve relevant sentences from an external corpus such as Wikipedia . The retrieved sentences are usually not interconnected , and their unstructured nature makes it inherently difficult for models to do complex reasoning over them ( Zhang et al. , 2018 ) . On the other hand , symbolic commonsense KGs such as ConceptNet ( Speer et al. , 2017 ) provide structured representation of the relational knowledge between concepts , which is of critical importance for effective ( multi-hop ) reasoning and making interpretable predictions . Therefore , recent advances ( Lin et al. , 2019 ; Feng et al. , 2020 ; Malaviya et al. , 2020 ; Bosselut & Choi , 2019 ) have focused on KG-augmented neural-symbolic commonsense reasoning — integrating the symbolic commonsense knowledge with the pre-trained neural language models such as BERT ( Devlin et al. , 2019 ) . One of the key challenges for KG-augmented commonsense reasoning is how to obtain relevant and useful facts for the model to reason over . These supporting facts are usually not readily available to the model and require explicit annotation by humans . Most existing works ( Lin et al. , 2019 ; Wang et al. , 2019b ; Lv et al. , 2020 ) follow heuristic procedures to extract supporting fact triples from KGs , e.g. , by finding connections between concepts mentioned in the question and answer . This simplified extraction process may be sub-optimal because the commonsense KGs are usually incomplete ( Min 1Code has been uploaded and will be published . et al. , 2013 ) and supporting facts could be missing . To mitigate this issue , Wang et al . ( 2020b ) finetune a language model to generate pseudo-paths ( i.e. , sequences of triples ) between question and answer concepts as evidence for the reasoning context ( question and answer ) . However , when two input concepts are not closely related , the generated pseudo-paths are often unreliable as it ’ s hard to connect two entities using a small set of predefined KG relations ( i.e. , limited expressiveness ) . Besides , since KGs are context-agnostic , both extracted facts and generated facts do not necessarily relate to the central topic of the reasoning context , yielding misleading facts for reasoning . Additionally , KGs themselves store noisy facts . To summarize , low coverage of KG facts , limited expressiveness of KG relations , wrong and uncontextualized facts make neural-symbolic integration of commonsense knowledge and pre-trained language models less reliable or generalizable . In this paper , we propose a novel KG-augmented commonsense reasoning model , named Hybrid Graph Network ( HGN ) , to address these issues . It leverages both extracted facts ( with high precision ) and continuous feature representations for generated facts ( with high recall ) to build a contextualized graph with learnable edge features , which overcome the low coverage and limited expressiveness issue of the KG . It then iteratively prunes unreliable and unrelated edges during model learning , leading to a superior graph structure for reasoning . Fig . 1 shows an illustrative example of the graph structure HGN has learned . Besides triples extracted from ConceptNet , e.g. , ( print , RelatedTo , use ) , HGN manages to ( 1 ) generate novel triples and ( 2 ) identify critical evidence triples , e.g. , ( print , Requires , use paper ) and ( paper , HasProperty , expensive ) , while pruning triples that are unhelpful for reasoning , e.g. , ( use , · , expensive ) . The final contextualized graphs created by our HGN are shown to be more useful for models to reason over . We summarize our contributions as follows : ( 1 ) We propose HGN , a KG-augmented commonsense reasoning model that overcomes the low coverage , limited expressiveness , wrong and uncontextualized facts issues of KGs . It jointly generates features for novel facts to complement extracted facts and learns the structure of the contextualized knowledge graph while reasoning over it . ( 2 ) We conduct extensive experiments on three commonsense question answering benchmarks and show consistent improvement over previous approaches . ( 3 ) We show our contextualized graph structures are more helpful for the question-answering process with user studies . 2 NEURAL-SYMBOLIC MODELS FOR COMMONSENSE REASONING . We focus on the task of commonsense question answering ( QA ) , while the proposed model can be easily adapted to other tasks that require commonsense reasoning skills ( e.g. , natural language inference ) . In the typical scenario of KG-augmented question answering , given a question q , the model is asked to select the correct answer from a set of candidate answers { ai } with the help of symbolic knowledge from an external knowledge graph G = { E , R , F } . Here , E , R , F denote the set of entities , relations , and facts , respectively . A fact takes the form of a triple ( h , r , t ) ∈ F , where h ∈ E is the head entity , t ∈ E is the tail entity , and r ∈ R is their relation . We approach the multi-choice QA problem by measuring the plausibility ρ ( q , a ) between the question q and each candidate answer a . The candidate answer with the highest plausibility score will be chosen as the model ’ s prediction . Fig . 2 illustrates the workflow of a typical neural-symbolic model architecture for question answering , which our proposed model fits into . The final score is predicted based on the neural encoding of unstructured reasoning context and symbolic graph knowledge . Neural Encoding of Reasoning Context . The text of the question and answer itself serves as strong unstructured evidence in evaluating their plausibility . Recent years have witnessed great success of pretrained language models ( PLMs ) ( Devlin et al. , 2019 ; Liu et al. , 2019 ) in a range of NLP tasks , including question answering ( Su et al. , 2019 ; Lukovnikov et al. , 2019 ) and natural language inference ( Zhang et al. , 2019 ; Wang et al. , 2020a ) . Similar to previous works , here we adopt a PLM parameterized by θtext to encode the question and answer pair into the statement vector : s = ftext ( [ q , a ] ; θtext ) . We use [ · , · ] to denote the concatenation of sentences or vectors . Modeling Symbolic Graph Knowledge . Commonsense knowledge graphs , as an external source of knowledge , can also contribute to context understanding and reasoning by providing relational knowledge between concepts related to the question and answer . For each question-candidate answer pair ( q , a ) , we build a directed graph G = ( V , E ) with adjacentcy matrix A ∈ { 0 , 1 } n×n , which is termed as the contextualized knowledge graph . It represents the relevant knowledge structures ( concepts and their relations ) from the external KG . G ’ s node set V includes concepts mentioned in the question and candidate answer pair ( q , a ) . Edges in G represent the relations between their connected nodes . G stores knowledge that is related to the context , and serves as the structured evidence for answering the question . Fig . 1 presents an example of the contextualized knowledge graph for the question and a candidate answer ( “ use paper ” ) . For a contextualized KG G with n nodes and m edges , we let V = { v1 , . . . , vn } . We denote the node feature vectors as { xi | vi ∈ V } and the edge feature vectors as { x ( i , j ) | ( vi , vj ) ∈ E } . We stack node feature vectors and edge feature vectors to get the node feature matrix X ∈ Rn×dv and the edge feature matrix Xe ∈ Rm×de , respectively . We denote the learnable parameters involved in the function which maps ( V , E ) to feature embeddings ( X , Xe ) as θgraph-emb . For encoding the contextualize knowledge graph G , we consider a general formulation of the graph encoder g = fgraph-enc ( X , Xe , A , s ; θgraph-enc ) , parameterized by θgraph-enc . For simplicity , we denote it as g = fgraph ( q , a , s ; θgraph ) where θgraph = { θgraph-emb , θgraph-enc } . To predict the plausibility score ρ ( q , a ) , we feed [ s , g ] , the concatenation of s and g , to a multilayer perceptron ( MLP ) with parameters θMLP . The output of the MLP is then passed through to a softmax layer to calculate the final probability ρ̂ ( q , a ) for choosing a candidate answer , shown as follows . ρ ( q , a ; θ ) = fMLP ( [ s , g ] ; θMLP ) ; { ρ̂ ( q , ai ; θ ) } = softmax { ρ ( q , ai ; θ ) } . ( 1 ) Here θ = { θtext , θgraph , θMLP } is the set of all learnable parameters . To derive the contextualized KG , most existing works perform heuristic graph extraction ( Lin et al. , 2019 ; Feng et al. , 2020 ) , while Wang et al . ( 2020b ) generate relational paths to connect question and answer concepts . They all assume a perfect graph structure and fix the adjacency matrix during training . In contrast , our proposed HGN starts with an initial graph structure with both extracted and generated edges , and iteratively refines the adjacency matrix during graph encoding . 3 JOINTLY LEARNING GRAPH STRUCTURES AND PARAMETERS . As illustrated in Fig . 2 , given a question and candidate answer ( q , a ) , we encode individual contextualized KG with a graph encoder fgraph , and then use the output graph vector g to estimate the plausibility of the question-answer pair . However , static graphs extracted from external KGs often suffer from limited coverage , making it hard for the model to collect and reason over adequate supporting facts . We solve the problem by considering both extracted facts and generated facts during graph initialization , resulting in a graph with “ hybrid ” edge features from complementary sources . For the generated facts , we directly use the continuous relational features output by q generator instead of decoding into relations and re-encoding with a lookup table for better expressivity . While incorporating generated facts improve the recall , evidence precision could be impacted . Besides , the processes of extracting and generating facts are context-agnostic , leading to irrelevant facts that could mislead reasoning . Therefore , we introduce learnable edge weights to control message passing during graph encoding . We further impose entropy regularization on the learned edge weights to encourage pruning noisy edges while performing reasoning . The overview of our graph module is shown in Fig . 3 . Starting from an heuristically extracted graph ( with adjacency matrix Aextract ) , we first generate edge features to enrich Aextract into a graph with fully-connected edges between question and answer concepts , denoted by an adjacency matrix A0 . Then we iteratively update the edge embeddings , the adjacency matrix and the node embeddings . The graph vector g is derived by pooling over node embeddings at the final layer . Formally , we denote the label for question-answer pair ( q , a ) as y , where y = 1 means a is the correct answer to q and y = 0 means a is a wrong answer . The overall objective of jointly learning graph structure and model parameters on a training set Dtrain is defined as follows : L ( θ ) = ∑ ( q , a , y ) ∼Dtrain [ Ltask ( ρ̂ ( q , a ; θ ) ) , y ) + β · Lprune ( AL ( q , a ; θtext , θgraph ) ) ] , ( 2 ) where θ = { θtext , θgraph , θMLP } is the set of all learnable parameters , β is a hyperparameter . AL represents the final graph structure after L layers ’ refinement ( §3.2 ) . L can be decomposed into Ltask for the downstream classification task and Lprune for graph structure learning with regularization . In the following subsections , we first introduce how we initialize the contextualized graph G with densified adjacency matrix A0 , node features X and hybrid edge features Xe . Next we show how we encode the graph as g = fgraph-enc ( X , Xe , A 0 , s ) and calculate Ltask for the classification task . Finally we show how we calculate the regularization termLprune based on the learned graph structure .

Summary: In this paper, the authors propose a new approach towards incorporating knowledge graphs (KG) into commonsense QA frameworks. KGs are helpful for adding structured "world" information, which neural-symbolic architectures can leverage to do commonsense reasoning, e.g., "what is the expensive resource in printing on paper?" (paper). In such architectures, however, the authors argue that KG quality is a large impediment (e.g., missing or incorrect edges, distracting nodes, etc). Therefore, they propose a "hybrid" KG-based model (accordingly named "Hybrid Graph Network") that jointly learns to refine/augment the graph structure while also optimizing it for inference performance.