Learning Contextualized Knowledge Structures for Commonsense Reasoning
Authors: Jun Yan
Mrigank Raman
Tianyu Zhang
Ryan Rossi
Handong Zhao
Sungchul Kim
Nedim Lipka
Xiang Ren
Venue:  (2021)

Abstract: Recently, neural-symbolic architectures have achieved success on commonsense reasoning through effectively encoding relational structures retrieved from external knowledge graphs (KGs) and obtained state-of-the-art results in tasks such as (commonsense) question answering and natural language inference. However, current neural-symbolic reasoning methods rely on quality and contextualized knowledge structures (i.e., fact triples) that can be retrieved at the pre-processing stage and overlook challenges such as dealing with incompleteness of a KG (low coverage), limited expressiveness of its relations, and irrelevant retrieved facts in the reasoning context. 
In this paper, we present a novel neural-symbolic approach, named Hybrid Graph Network (HGN), which jointly generates feature representations for new triples (as complement to the existing edges in the KG), determines relevance of the triples to the reasoning context, and learns graph model parameters for encoding the relational information. Our method learns a compact graph structure (comprising both retrieved and generated edges) through filtering edges that are unhelpful to the reasoning process. We show marked improvements on three commonsense reasoning benchmarks and demonstrate the superiority of the learned graph structures with user studies.

Under review as a conference paper at ICLR 2021
LEARNING CONTEXTUALIZED KNOWLEDGE STRUC- TURES FOR COMMONSENSE REASONING
Anonymous authors
Paper under double-blind review
ABSTRACT
Recently, neural-symbolic architectures have achieved success on commonsense reasoning through effectively encoding relational structures retrieved from exter- nal knowledge graphs (KGs) and obtained state-of-the-ar results in tasks such as (commonsense) question answering and natural language inference. How- ever, these methods rely on quality and contextualized knowledge structures (i.e.. fact triples) that are retrieved at the pre-processing stage but overlook challenges caused by incompleteness of KG, limited expressivity of its relations, and re- trieved facts irrelevant to the reasoning context. In this paper, present novel neural-symbolic model, named Hybrid Graph Network (HGN), which jointly gen- erates feature representations for new triples (as complement to existing edges in the KG), determines the relevance of the triples to the reasoning context, and learns graph module parameters for encoding the relational information. Our model learns a compact graph structure (comprising both extracted and gener- ated edges) through filtering edges that unhelpful to the reasoning process. We show marked improvement on three commonsense reasoning benchmarks and demonstrate the superiority of the learned graph structures with user studies. [
INTRODUCTION
Commonsense knowledge is essential for developing human-level artificial intelligence systems that understand and interact with the real world. However, commonsense knowledge is assumned by humans and thus rarely written down in text corpora for machines to learn from and make inferences with. Fig. T shows example in popular commonsense reasoning benchmark named Common- senseQA (Tammor et al.) 2019). The knowledge about the relations between concepts, e.g-, the fact triple (print, Requiress res, use paper), is not explicitly given in the question and answer. With- out important background knowledge as clues, natural language understanding (NLU) models may fail to answer such simple commonsense questions that are trivial to humans.
Current commonsense reasoning models can be classified into retrieval-augmenter methods (Baner- jee al. 2019; Pan al., 2019) and KG-augmented methods (Wang et al., 2219b;; KKaanipatti et al. 2020). Rerrivallaaumented methods retrieve relevant sentences from an external corpus such as Wikipedia. The retrieved sentences are usually not interconnected, and their unstructured nature makes it inherently difficult for models to do complex reasoning over them (Zhang et al., 2018). On the other band, symbolic commonsense KGs such as ConceptNet (Speer et al. provide struc- tured representation of the relational knowledge between concepts, which is of critical importance for effective (multi-hop -hop reasoning and making interpretable predictions. Therefore, recent advances (Lin et al.) 2019 Feng al. 22200; Malaviya et al. 2020; Bosselut & Choil 2019) have focused on KG-augmented neural-symbolic commonsense reasoning integrating the symbolic commonsense knowledge with the pre-trained neural language models such as BERT (Devlin al. 2019).
One of the key challenges for KG-augmented commonsense reasoning is how to obtain relevant and useful facts for the model reason over. These supporting facts are usually not readily available to the model and require explicit annotation by humans. Most existing works (Lin et al. 2019 Wang
al. 2019b Lv et al. 2020) follow heuristic procedures extract supporting fact triples from KGs, e.g., by finding connections between concepts mentioned in the question and answer. This simplified extraction process may be sub-optimal because the commonsense KGs are usually incomplete (Mi
Code heen uplonded published.

---

Under review as a conference paper at ICLR 2021
Contextualized Knowledge Graph for (<Question>, "usc paper")
paper
Requires
print
RolstafTo
RelriedTo
use
DietnceFrom HasPeppory
expensive
extracted fact (pruned)
extracted fact (kept)
generated fact (kept)
paper
generated fact (pruned)
Question>> Printing printer can get expensive because does what?
<Candidate A nswers>>
A. explode B. use paper C. store information D. queue E. noise
Commonsense question answering augmented with external graph knowledge. Underlined words and phrases are recognized concepts. To correctly answer this question, it's desirable that the model has access commonsense knowledge like (print, Requires. expens ve). which not presented the context. This calls the integration of contextualized common- sense knowledge.
Figure 1:
paper), (paper, SlasPropertt
et al. 2013) and supporting facts could be missing. To mitigate this issue, Wang et al. (2020b) fine- tune a language model to generate pseudo-paths (i.e., sequences triples) between question and answer concepts as evidence for the reasoning context (question and answer). However, when two input concepts are not closely related, the generated pseudo-paths are often unreliable as it's hard to connect two entities using a small set of predefined KG relations (i.e., limited expressiveness). Besides, since KGs are context-agnostic, both extracted facts and generated facts do not necessarily relate to the central topic of the reasoning context, yielding misleading facts for reasoning. Ad- ditionally, KGs themselves store noisy facts. To summarize, low coverage of KG facts, limited expressiveness of KG relations, wrong and uncontextualized facts make neural-symbolic integration of commonsense knowledge and pre-trained language models less reliable or generalizable.
In this paper, we propose novel KG-augmented commonsense reasoning model, named Hybrid Graph Network (HGN), to address these issues. It leverages both extracted facts (with high preci- sion) and continuous feature representations for generated facts (with high recall) to build a con- textualized graph with learnable edge features, which overcome the low coverage and limited ex- pressiveness issue of the KG. It then iteratively prunes unreliable and unrelated edges during model learning, leading to superior graph structure for reasoning. Fig. [ shows an illustrative example of the graph structure HGN has learned. Besides triples extracted from ConceptNet, e.g., (print, Rela RelatedTo, use), HGN manages to (1) generate novel triples and (2) identify critical evidence triples. e.g., (print, Requires, use paper) and (paper, HasProperty, expensives while pruning triples that unhelpful for reasoning, e.g., (use, expensive). The final con- textualized graphs created by our HGN are shown to be more useful for models to reason over.
We summarize contributions follows: (1) We propose HGN, a KG-augmented commonsense reasoning model that overcomes the low coverage, limited expressiveness, wrong and uncontextu- alized facts issues of KGs. It jointly generates features for novel facts to complement extracted facts and learns the structure of the contextualized knowledge graph while reasoning over it. (2) We conduct extensive experiments consistent improvement over previous approaches. (3) We show our contextualized graph structures are more helpful for the question-answar process with user studies.
three commonsense question answering benchmarks and show
NEURAL-S SYMBOLIC MODELS FOR COMMONSENSE REASONING
We focus the task of commonsense question answering (QA), while the proposed model can be easily adapted to other tasks that require commonsense reasoning skills (e.g., natural language inference). In the typical scenario of KG-augmented question answering, given question q. the model asked to select the correct answer from a set candidate answers {az} with the help of symbolic knowledge from external knowledge graph = {3, R, F). Here, E, R, denote the set of entities, relations, and facts, respectively. A fact takes the form of a triple (h, r,t) t) E F. where h E is the head entity, t & is the tail entity, and R is their relation.
We approach the multi-choice QA problem by measuring the plausibility p(g. a) between the ques- tion y and each candidate answer a. The candidate answer with the highest plausibility score will be
2

---

Under review as a conference paper at ICLR 2021
Contextualized KG:
Graph Encoder
G
Graph Inttiellzaiion
Text Encoder
(q. a]
concal.
MLP
p(q.a)
Figure 2: Architecture of typical neural-symbolic model for commonsense reasoning.
chosen as the model's prediction. Fig. 2 illustrates the workflow of a typical neural-symbolic model architecture for question answering, which our proposed model fits into. The final score is predicted based on the neural encoding of unstructured reasoning context and symbolic graph knowledge.
Neural Encoding of Reasoning Context. The text of the question and answer itself serves as strong unstructured evidence in evaluating their plausibility. Recent years have witnessed great
success of pretrained language models (PLMs) (Devlin et al. 2019 Liu et al. 2019) in ränge of NLP tasks, including question answering (Su et al. [
Lukovnikov et al. 2019) and natural
language inference (Zhang et al., 2019 Wang et al. 2020a). Similar to previous works, here we
adopt PLM parameterized by 1ext to encode the question and answer pair into the statement vector: s = Jee((g. a|: Oeeu). We use
[:-]] to denote the concatenation of sentences or vectors.
Modeling Symbolic Graph Knowledge. Commonsense knowledge graphs, as an external source of knowledge, can also contribute to context understanding and reasoning by providing relational knowledge between concepts related to the question and answer. For each question-candidate answer pair (q, a). we build a directed graph G = (V, E) with adjacentey matrix A E {0, 1|mx*, which termed as the contextualized knowledge graph. It represents the relevant knowledge structures (concepts and their relations) from the external KG. G's node set includes concepts mentioned in the question and candidate answer pair (g, a). Edges in G represent the relations between their connected nodes. G stores knowledge that is related to the context, and serves as the structured evidence for answering the question. Fig. [ presents an example of the contextualized knowledge graph for the question and a candidate answer ("use paper").
For contextualized KG G with n nodes and edges, we let V {ui...., Un We denote the node feature vectors as {x; Us E V} and the edge feature vectors as {x(i,) (0), U2) E E}. We stack node feature vectors and edge feature vectors to get the node feature matrix X E RRadd and the edge feature matrix X. IR mgd,,, respectively. We denote the learnable parameters involved in the function which maps (V, E) to feature embeddings (X, Xz) as O graph-emm- For encoding the contextualize knowledge graph G. we consider a general formulation of the graph encoder g = gorpheenc (X,X., Xx, A, s; O graph-enc), parameterized by 0 graph-enc- For simplicity, we denote it as g graph (q. a, S: O graph) where 0 graph = {0 (0graph-emt O graphhenne) : To predict the plausibility score p(g, a). feed (s. g). the concatenation of and g. to multilayer perceptron (MLP) with parameters M The output of the MLP is then passed through to a softmax layer to calculate the final probability p(g. a) for choosing a candidate answer, shown follows.
p(q, a; 9) fMLP((S ([s g); OMLP);
{p(q. oz 0)) softmax (plx, a; 0)}.
(1)
Here = (O graph M is the set of all learnable parameters.
To derive the contextualized KG, most existing works perform heuristic graph extraction (Lin et al., 2019 Feng al.) 2020), while Wang et al. (20206) generate relational paths to connect question and answer concepts. They all assume a perfect graph structure and fix the adjacency matrix during training. In contrast, our proposed HGN starts with an initial graph structure with both extracted and generated edges, and iteratively refines the adjacency matrix during graph encoding.
JOINTLY LEARNING GRAPH STRUCTURES AND PARAMETERS
As illustrated in Fig. 2 given a question and candidate answer (q, a). we encode individual con- textualized KG with graph encoder Sgmph, and then use the output graph vector to estimate the plausibility of the question-answar pair. However, static graphs extracted from external KGs often suffer from limited coverage, making it hard for the model to collect and reason over adequate sup- porting facts. We solve the problem by considering both extracted facts and generated facts during graph initialization, resulting in a graph with "hybrid" edge features from complementary sources.
3

---

Under review as a conference paper at ICLR 2021
Axtracct
A
A'
A'
milcultt repilcriaaionn Grune (A') term tinal layer pool overnoda
generate Iytrid odge fostims
embedilins at Linl layer" g
[q, s]
H H
H' H
H mode embeddings at layer
H: edge embeddings at layer!
1=(+1
A adj aceecy matns layer
Figure 3: Overview of our HGN's graph module. jointly learn the graph structure and network parameters. Darkness of edges indicate their weights. Red variables are updated in the previous step.
For the generated facts, we directly use the continuous relational features output by q generator in- stead of decoding into relations and re-encoding with lookup table for better expressivity. While incorporating generated facts improve the recall, evidence precision could be impacted. Besides, the processes of extracting and generating facts context-agnostic, leading irrelevant facts that could mislead reasoning. Therefore, we introduce learnable edge weights to control message pass- ing during graph encoding. We further impose entropy regularization the learned edge weights to encourage pruning noisy edges while performing reasoning.
The overview of our graph module is shown in Fig. Starting from heuristically extracted graph (with adjacency matrix A extract,, we first generate edge features to enrich extract into a graph with fully-connected edges between question and answer concepts, denoted by an adjacency matrix A". Then we iteratively update the edge embeddings, the adjacency matrix and the node embeddings. The graph vector g is derived by pooling over node embeddings at the final layer.
Formally. we denote the label for question- question-answer answer pair (g, a) as where y 1 means a is the correct answer q and y means a is wrong answer. The overall objective of jointly learning graph structure and model parameters on training set Duain is defined as follows:
(())
Σ
[Crakk (Ā(g, a; 0)). + B Cprune (A (g, a; ttext, 2 @graph))
(2)
(9.0,a)~
where 0 (0 lext: 2 graaph: @MLP} is the set of all learnable parameters, B is hyperparameter. A represents the final graph structure after L layers' refinement (33.2). L can be decomposed into Lask for the downstream classification task and Lynune for graph structure learning with regularization.
In the following subsections, we first introduce how we initialize the contextualized graph G with densified adjacency matrix A node features X and hybrid edge features X.- Next show how we encode the graph as = fgraph-ene (X,X,, Xe, A s) and calculate Lusk for the classification task. Finally we show how we calculate the regularization term Lprane based the learned graph structure.
3.1 GRAPH INITIALIZATION
Node Set and Node Features. To effectively acquire knowledge from G, we need to ground con- cepts in (q, a) the entity set of G. A concept mention defined as a text span that corresponds to an entity in E. We perform string matching based on the concept vocabulary of ConceptNet (Speer al. 2017) to identify concept mentions q and a. The sets of recognized question and an- swer concepts are denoted as v2 = { and V = } For each node 2ú E V where V 1 U VA, we use its corresponding entity embedding e,." the node feature vector X;.
Edge Feature Generator. To account for the limited coverage issue of KGs, we build an edge feature generator (ge((:-) that takes subject concept and an object concept input, and gen- erates a vector that encodes their relation. This can be modeled as a knowledge graph completion (KGC) task. Unlike conventional KGs, commonsense KGs are much sparser (Malaviya al., 2019). which poses challenges for standard smbedding-basee approaches (Ynng et 20144) Dettmers et al.) 2017]. Recent works (Malaviya al. 2019 [ooselut et al. 2019) show that pretrained language models can effectively tackle the sparsity challenge with well-learmed concept semantics captured during large-scale pretraining. Also, pretrained language models themselves have proven to possess certain commonsense knowledge (Davison et al. 2019 Petroni et al., 2019). These features make pretrained language models a more ideal choice for the commonsense KG completion task. Inspired by recent success on adopting sentence generation formulation for commonsense knowledge com- pletion (Bosselut et al.) 2019 Wang et al.] 20206), as an implementation, we define a sentence generation task where the model is asked generate the relation tokens given the subject and the
4

---

Under review as a conference paper at ICLR 2021
object. Specially, each fact (h, r, t) is associated with sentence in the "prompt- format: h; $; 8; h;;;;;; where h, P, are the word sequence of h, r, $ is the separator used by GPT-2. We convert all facts from ConcepiNet to this format and finetune a GPT-2 on these sentences. Then we can use the finetuned GPT-2 to generate sentence describing the relation between a given pair of subject and object. We take the hidden states during generation to compute the feature vector for better expressivity. More implementation details can be found in A An alternative solution is pro- posed in Wang et al. (2020b), where a GPT-2 is trained to generate a relational path connecting the given subject and object. The relational path has been proved store rich information for inferring the relation (Neelakantan et al. 2015] Das et al. (017).
Edge Set and Hybrid Edge Features. We build folly-connected edges between question concepts and answer concepts as they model the interactions between the question and answer, which serve as discriminative features in evaluating their plausibility. The adjacency matrix A row-i column- j element is if (M,, v2) E E (V' x V²) U (V* x VQ), and 0 otherwise. Note that for a large portion of (vy, 04) E E, there may not be any fact from the knowledge graph that describes their relation. We therefore use the learned edge feature generator fgen(".' to generate their feature vectors. For an edge connecting concept pairs that are adjacent in g. can be mapped to a KG relation, and we use the relation embedding as the feature vector. Formally, for any (11, U2) E E, the edge feature vector ((i.j) is calculated in a hybrid way as:
Eir E R, (. EFF x(()) Sadupe (Sgen (1i, 0;)), otherwise.
(3)
efl is the relation embedding for KG relation r. faduge(.) is an MLP parameterized by ndapt, which is used to transform the generated feature vector into the same space efa!
Since we freeze the weights of the edge feature generator fgen (;.), learnable parameters involved in the graph initialization of our HGN is O grapb-cmb = {OOctt:
cat: O O adapec Here 0 {eet c E E) is
the set of entity embeddings and 0 (e r E R is the set of relation embeddings.
3.2 GRAPH REASONING
We obtain an unweighted graph G with node and edge features in Although we use an edge feature generator to densify the connections between question and answer concepts, we are still facing the challenges of noisy edges, specifically unreliable edges (edges with wrong or low-quality attributes) and unrelated edges (edges irrelevant to answering the question), G. Therefore, we propose to jointly refine the structure of the contextualized graph while performing reasoning.
Reasoning with Weighted Graph. uous relaxation of the problem. Specifically, we generalize the unweighted graph to a weighted one and then learn to reweight all edges during reasoning. We build our graph reasoning module based on the formulation of Graph Networks (GNs) (Battaglia et al. 2018) by instantiating the layerwise node-to-edge (r e) and edge-to edge-to-node (e - e) message passing functions. The edge weight indi- cates the helpfulness of an edge in reasoning and is used as rescaling factor to control the message flow on this edge. Formally, the propagation rule at layer / is defined as:
To learn a graph structure for reasoning, consider contin-
v v e blo = fime ([n) [n] 1n b(
sotts) = f ([bta)is)) A(os)) = Eietie (a.))
e e u. u/o = ft. ( :bi = fi-mo
Σ iENg
(4) N2 is the set of uy's incoming neighbors, Pimee SussseSSu Ju and P8-0 are MLPs, hk.)) X(1)), h² x;- In node-to-edge message passing, for each edge (v;, vU) E E, we calculate its updated edge embed- ding (i.)) which encodes the corresponding fact. We also assign to an unnormalized score w((.)) which measures both the validness of the fact itself (captured by b/7))) and how helpful it is to the reasoning context (captured by s). The edge score is globally normalized across all edges to become the edge weight (...)) so that edges with low scores will be softly pruned by receiving close-to- zero weight. We choose global normalization instead of local normalization (normalization within
5

---

Under review as a conference paper at ICLR 2021
each node's neighborhood) like Graph Attention Network (GAT) (Veličković et al. 2017) because local normalization assumes at least one edge should be helpful in a node' 's incoming neighborhood, which is not true in our situation. For example, for the distracting or wrongly-grounded grounded concepts, none of their connected edges could be helpful and all edges in the neighborhood should be pruned. Those noisy nodes should be softly excluded from message passing rather than still receive message from a weighted combination of their neighbors. We also empirically compare our proposed model with a variant using GAT-like edge attention 44.3
In edge-to-node message passing, we calculate the message vector u(a.)) based on the start node and edge embeddings. We use the edge weight to rescale the message vector and each node aggregates messages from incoming neighbors to update its node embedding.
Once we get the final node embeddings after L layers' message passing, we aggregate them into graph-level representation through a graph pooling operation. As different nodes can have distinct informativeness, we employ attention mechanism, where nodes are assigned different importance, to obtain the graph encoding g: a; sW h g EinLev Ejeeer an h/. Here Wau is
ent
learnable matrix for calculating each attention score a, for node vt., Given g, we calculate p(g, a) using Eq. We adopt the cross-entropy loss for the main classification task:
Lask (p(g, a; 0)), v) = y log p(q, a; 0).
(5)
Learning to Prune Edges with Entropy Regularization. To encourage the model to take decisive pruming steps the graph structure, add regularization term to the loss function to penalize non-discriminative edge weights. In an extreme case, a blind model will assign the same weight to all edges and G is degenerated into unweighted graph, where usually big number of noisy edges are mixed with a small number of helpful edges. Therefore, to guide the model to discriminate the helpful edges in the reasoning process, we minimize the entropy of the edge weight distribution as an auxiliary training objective. The motivation is that the information entropy can be used to measure the informativeness of the edge weight predictions. A lower entropy, caused by a skewed distribution. means the model is actively incorporating more priors (e.g. the plausibility of its corre- sponding fact and the relatedness the question- question-answer answer pair) into edge weight prediction. Formally, the entropy regularization term of G is calculated as:
Cprume (A L (q, a; QuxI: e @graph)
Σ (L.)):(LL,)))EE
(1i) log A
(6)
We add the penalty term the downstream classification loss so that the graph structure can be jointly learned with graph reasoning. We train our model end-to-end by minimizing the overall objective function ((0) in Eq. 2 using RAdam (Liu et al. 2020) optimizer.
4
EXPERIMENTS
4.1
EXPERIMENT SETUP
We evaluate our proposed model on three multiple-choice commonsense QA datasets: Common- senseQA (Talmor et al. 2019), OpenbookQA (Mihaylov et al. 2018) and CODAH (Chen et al., 2019) (details in B We use ConceptNet (Speer et al. 2017). a commonsensense knowle knowledge graph, as G. For text encoder fixtt. we experiment with BERT-base, BERT-large (Devlin et al., 2019) and RoBERTa (-large) (Liu et al. 2019) to validate our model's effectiveness over different text en- coders. For OpenbookQA, retrieving related facts from the openbook plays an important role in boosting the model's performance. Therefore, we also build our graph reasoning model on top of a eetrieval-augmented method "Aris."RoBERTa" this way, we can study if strong retrieval- augmented methods could still benefit from KG knowledge and our reasoning framework.
4.2 COMPARED METHOOSS
We compare model with series of KG-augmented methods:
Models Using Extracted Facts: RN (Santoro et al.) 2017) builds the graph with the same node set as our method but extracted edges only. The graph vector is calculated g Pool([MLP([S ((s))((( x,l)
0,)) E F)). GN (Battaglia et al.] 20188) presents a general formulation of GNNs. We instantiate it with the layerwise propagation rule defined in Eq. 4 It

---

Under review as a conference paper at ICLR 2021
Methods
BERT-BAS Base
BERI-Large
RoBERTa
60% Train
100% Train
60% Train
100% Train
60% Train
100% Train
LM Pinnttniig
52.06 (110.72)
53.47 (10.7)
$2.30 (110.16)
55.39 (100.0)
65,56 (.10.76)
68.69 (100.56)
et. 2017
(100.00
56.20 (10.45)
54.23 (100.28)
58.46 (1.0.71)
(.0.28)
70.08 (10.21)
RN + Link Predieton
53.96 (210.56)
56.02 (=:0.55)
66.29| 200
69,33 (20.98)
RGCN Schlichtkrull
2018b
())
54.50 (+0.56)
54.71 (+0.37)
57.13 (=0.36)
68.33 (=0.85)
68.41 (+0.60)
GN (Rahagua el al. 2018
53.67 (=0.45)
55.65 (+0.51)
54.78 (+00661)
57.81 (+00677
68.78 (++067)
71.12 (+0.45)
GeonAun Waay
20119a
51.36 (1.0.8))
54.41 (10.50)
54.96 (10069)
56.94 (1.0.77)
68.09 (110.63)
69.88 (10.47)
KagNet (Li erai 2019
56.19
57.16
MHGRN (Fenger 2020
(=0.49))
56.23 (+0.82)
56.76 (+0.21)
59.85 (=00.66)
68.8 (=1106)
71.11 (=0.81)
PathGenerhnoo (Wang ale 2020b
45.47 (+0.92)
57.21 (+:0445)
68.65 (=00.02)
71.55 (+0.99)
HGN (wlo edge weights) HGN
$4.45
(9339)
56.56 (110.7)
55.67 (20065)
58.89 (2:0.45)
70.01 (=10.91)
72.09 (20.87)
$5.39 (0034)
57.82 (+0.23)
57.23 (+0.56)
60.43 (=00544)
70.34 (=0.79)
72.88 (=0.83)
Table 1: Accuracy on Qommonsense inhouse test set. We use the inhouse split as Lin et al. 2019). Some of the baseline results are reported by Feng et al. (2020) and Wang et al. Mean and standard deviation of four runs are presented for all models except KagNet.
2020b)
Methods
RoBERTa
AristoRotEERTa
LM Finctoning
64,80 (::2.37)
1.666
Methods
Text Encoder
Test Acc
(antorr 2017
63.65 (1:2.31)
75.35 (1:1.39)
UnifiedQA KKashabi al. 2020
TS T3
87.2
66.30 (+0048)
(+
TS KB
85.4
RGCN (Schlichskrall at 2018b
62.45 (+11.57)
(+-2.53)
(Raieel
2020
X3.2
GN Bohnggia of E. 2015
66.20 (+:2.14)
77.25 (=:0.91)
ParhCemeence Wang al 2020hh
AristoAlbert
81.8
GaoiAti (Wiis 91
64.75
(++..21)
HGN (ours)
AristoRoBERIa
81.4
MHGRN (Feigre
66.85 (+11.19)
77,75 (++0.38)
AristoRoBERTs KR
Trisn
81.0
PathGenesanrr Waliger deme 20206
68.40 (+b0.31)
80.05 (=b0.68)
MHGRN Peng 2020
AriooRoBERR
80.6
PathGenerace
AristoRBERRa
80.2
HGN (wlo edge weights)
67.20 (+:2.07)
78.45 (#-0.46)
KF + SIR (Bankejer & Bara 2020
RoBERTa
80.2
HGN
69.00 (1.0.55)
ArisoRoBEK12
AristoRBBERTa
80.2
Table 2: Test accuracy on OpenbookQA. Some of the baseline results are reported by Feng et al. (2020) and and[aang et (2020b). Mcan and standard deviation of four runs are presented for all models.
Table 3: Leaderboard of OpenbookQA Our HGN ranks first among all submissions using AristoRoBERTa as the text encoder.
differs from our HGN in that: (1) it only considers extracted edges; (2) all edge weights are fixed to 1. MHGRN (Feng et al.) 2020) generalizes GNNs with multi-hop message passing. Descriptions for RGCN, GconAttn, and KagNet can be found in (
Models Using Generated Facts: RN + Link Prediction differs from RN by only considering the generated relation (predicted using TransE (Bordes et al.) 2013)) between question and answer con- cepts. PathGenerator FI3) (Wang et al. (0206) learns a path generator from paths collected through random walks on the KG. The learned generator is used to generate paths connecting question and answer concepts. Attentive pooling is used to derive the graph vector given a set of path embeddings.
Our Model's Variant: HGN (w/o edge weights) reasons over an unweighted graph with hybrid features, which means edge weights are fixed to during training.
4.3 RESULTS
Performance Comparisons. Tables 1 2 and 4 show performance comparisons between our models and baseline models on Commonsense OpenbookQA, and CODAH respectively. Our HGN shows consistent improvement over baseline models all datasets except that it achieves the second best performance on OpenbookQA with BriRtoRoBERTa as the text encoder. We also submit our best model OpenbookQA's leaderboard Our model ranks the first among all models using Arist as the text encoder demonstrating the effectiveness of our proposed model. As a comparison, most baseline models fail to achieve further improvement over it. That may be because BoERia has access to knowledge collected through retrieval, which makes it difficult to further benefit from KG knowledge if weak reasoning approach is adopted. The improvement
2 "PathGenerator is a contemporancous work that learns a edge feature generator based on multi-hop paths, which has greater expressive power our (1-hop) edge feature generator. Our reasoning framework compatible with any implementation fyn. and we will build on PuihCenerrauur in our fulure experiments.
3 We choose PG-Global the representative variant for PathGonerator performs better than PG-Local. PG-Full is ensemble model
of PG-Gloaal and RN, so we don't 'tn't consider it in our comparisons.
We tried more than 4 seeds for the leaderboard submission, which different from the setting of Table
STS parameters, making impractical for to finetune available computational resources.
7

---

Under review as a conference paper at ICLR 2021
Commonaarastt Pecart argel
03
HRN
E3.00
12.04
Firetuning
UM Rinetuningg
MARGON
90
20
60
100
Thais
Aigromatia
ve glss
(a) CommonsenseQA.
(b) OpenbookQA.
(c) Ablations on model variants.
Figure 4: Ablation studies. (a)(b) Performance of HGN and baseline models with different amount of training data; (c) Performance of different model variants.
Methods
BERT-large
RoBERTa
Contextunlized Graph
GN (Arera))
HGN ()
LM Finetuning
65.89
83,20
Numher of Edges
3.65 (+2,73)
4.38 (+-3
RN Santorn et 2017
Number Valid
(=:155)
RGCN Iscrreniee al 20186
65.27
82.24
Percentage of Valid Edges
71.64%
78.51%
GN (Balngum
65.89
82.24
Average Helpfulaess Score of Edges
0.90 (20.550)
1.16 (200.11)
MHGGN (Fange
66.17
Prune Rute
77.13%
HGN (win edge weights)
66.88
83.25
HGN
66.71
83,75
Table 5: User studies on learned graph structures. 30 pairs of contextualized graphs output by GN and HGN evaluated by 5 annotators.
Table 4: Test accuracy on CODAH. We use the
same 5-fold cross validation as Yang et al. (220).
on CODAH is less significant compared the other two datasets. As Chen et al. (2019) suggest, questions in CODAH mainly target commonsense reasoning about quantitative, negation and object reference. this case, relational knowledge provided by ConceptNet may only offer limited help.
From the results of GN and our model variants, we can see "HGN (w/o edge weights)" consistently outperforms GN, which means generated facts could help reasoning, and suggests that current KG- augmented models can be improved with more complete KG. Our HGN further improves over "HGN (w/o edge weights)", indicating the effectiveness of conducting context-dependent pruning.
Training with Less Labeled Data. Fig. 4 (a)(b) show the results of our model and baseline mod- els when trained with /09/44%%/00///0%///// OpenbookQA. Our model gets better test accuracy under all settings. The improvement over the knowledge-agnen baseline (LM Finetuning) is more significant with less training data, which suggests that incorporating external knowledge is more helpful in the low-resource setting.
of the training data on CommonsenseQA and
Study on More Model Variants, To better understand the model design, we experiment with two more variants CommonsenseQA and OpenbookQA. GN all-generation doesn't 'ttt consider tracted facts and instead generate edge features between all question and answer concepts. HGN w/o statement vector doesn' 't consider in Eq. 4 which isolates the graph encoder from the text encoder. HGN w/ local attention replaces our edge weight design with local attention mechanism similar GAT (details in (D. Fig. (c) shows the results of the ablation study. Comparing "GN all- generation"" with "HGN w/o edge weights", we can conclude that extracted facts play an impor- tant role in HGN and can't be replaced by generated features. The high precision of extracted facts is still desirable even if we have model to generate relational edges. Comparing "HGN w/o state- ment vector" with "HGN", we find that accessing context information also important for graph reasoning, which means information propagation and edge weight prediction should be conducted in a context-aware manner. "HGN w/ local attention" outperformed by HGN, suggesting that "global edge weight" is more appropriate design choice than "local edge attention".
4.4 USER STUDIES ON LEARNED GRAPH STRUCTURES
To assess our model's ability to refine the graph structure, compare the graph structure before and after being processed by HGN. Specifically, we sample 30 questions with its correct answer from the development set of CommonsenseQA and ask 5 human annotators to evaluate the graph output by GN (with adjacency matrix A extract and extracted facts only) and our HGN (with adjacency matrix A L.. We manually binarizing A by removing edges with weight less than 0.01.
8

---

Under review as a conference paper at ICLR 2021
Given graph, for each edge (fact), annotators are asked to rate its validness and helpfulness. The validness score is rated as a binary value in a vontext-agnostic way: 0 (the fact doesn't 't make sense), 1 (the fact is generally true). The helpfulness score measures if the fact is helpful for solving the question and is rated on a to 2 scale: 0 (the fact is unrelated to the question and answer), (the fact is related but doesn't directly lead to the answer), 2 (the fact directly leads to the answer). The mean ratings for 30 pairs of (GN, HGN) graphs by 5 annotators are reported in Table 5
We also include
another metric named "prune rate" calculated as: 1- # edges in binarized A
# #ddges in A which measures the portion of edges that are assigned very low weights (softly pruned) during training and is only applicable to HGN. The Fleiss' Kappa (Fleiss) 1971) is 0.51 (moderate agreement) for validness and 0.36 (fair agreement) for helpfulness. The graph refined by HGN has both more edges and more valid edges compared to the extracted one. The refined graph also achieves higher helpfulness score. These all indicate that our HGN learns a superior graph structure with more helpful edges and less noisy edges, which is the reason for performance improvement over previous works that rely on extracted and static graphs. Detailed cases can be found in C
5 RELATED WORK
Commonsense Question Answering. Answering commonsense questions is challenging because the required commonsense knowledge is neither written down in the context nor held by pretrained language models. Therefore, many works leverage external knowledge to obtain additional evi- dence. These works can be categorized into IR-augmented methods, where evidence is retrieved from text corpora, and KG-augmented methods, where evidence is collected from KGs. Lv et al. 2020) demonstrate that IR-based evidence and KG-based evidence are complementary to each other. Although adding IR evidence can lead to further performance improvement, we only fo- cus on the challenges of KG-augmented methods in this paper. Literature in this domain mainly studies how to encode the contextualized subgraph extracted from KG. For example, Lin et al. (2019) propose a model comprised of GCN and LSTM account for both the global graph struc- ture and local paths connecting question concepts and answer concepts. Ma et al. (2019) use BERT to generate the embedding for the pseudo-sentence representing each edge and then adopt the at- tention mechanism to aggregate edge features as the graph encoding. Crucial difference is that they assume static graph and there's 's no operation on enriching or denoising the graph structure, While Wang et al. (2020b) also complete the contextualized graph with a path generator, they still reason over the static graph and neglect the noise introduced during generation.
Graph Structure Learning. Works that jointly learn the graph structure with the downstream task can be classified into two categories. One line of works directly learn an unweighted graph with desired edges for reasoning. KKipf et al. (2018) and Franceschi al. (2019) sample the graph structure from a predicted probabilistic distribution with differentiable approximations, Norcliffe- Brown al. (2018) calculate the relatedness between any pair of nodes and only keep the top-k strongest connections for each node to construct the edge set. Sun et al. (2019) start with a small graph and iteratively expand it with a of retrieving operations. The other line of works consider weighted graph with all possible edges and softly filter out the noisy ones by downweighting them. An adjacency matrix with continuous values is incorporated into message passing. [Tang et al. (2019) and Yu et al.] (2019) use heuristics to regularize the learned adjacency matrix. Hu jet al. (2019) consider the question embedding for predicting edge weights. Our HGN falls into the second category and therefore avoids information loss caused by hard pruning and approximation. Our uniqueness is that we construct the graph with hybrid features based on extracted and generated facts and we let node features, edge features, edge weights, and the global signal (statement vector) collectively determine the evolution of the graph structure. These empower our model with greater capacity and flexibility for (G-augmented QA.
6 CONCLUSION
In this paper, we propose a neural-symbolic framework for commonsense reasoning named HGN. To address the issues with missing facts from external knowledge graph and noisy facts from the contextualized knowledge graph, our proposed HGN jointly generates features for new edges, refines the graph structure, and learns the parameters for graph networks. Experimental results and user studies demonstrate the effectivvness of our model. In the future, we plan to incorporate open relations in graph initialization, which are more expressive than predefined KG relations. We also plan to study how make the fact generation or extraction process aware of the reasoning context.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Pratyay Banerjee and Chitta Baral. Knowledge fusion and semantic knowledge ranking for open domain question answering. arXiv preprint arXiv:2004.02 10 2020.
Pratyay Banerjee, Kuntal Kumar Pal, Arindam Mitra, and Chitta Baral. Careful selection of knowl- edge to solve open book question answering. In Proceedings of ACL, pp. 6120-6129, 2019.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Banchez-Gonzaler Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 100.11006.1111, 2018.
Antoine Bordes, Nicolas Usumier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances neural information processing systems, pp. 2787-2795, 2013.
Antoine Bosselut and Yejin Choi. Dynamic knowledge graph construction for zero-shot common- sense question answering. arXiv preprini 1rX1v:191..03876 2019.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. Comet: Commonsense transformers for automatic knowledge graph construction. In Pro- ceedings of the 57th Anmual Meeting of the Association for Computational Linguistics, pp. 4762- 4779, 2019.
Michael Chen, Mike D' D'Arcy, Alisa Liu, Jared Fernandez, and Doug Downey. Codah: An adversarially-authored question answering dataset for common sense. Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, pp. 63-69, 2019.
Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. Chains of reasoning over entities, relations, and text using recurrent neural networks. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 132 141, 2017.
Joe Davison, Joshua Feldman, and Alexander M Rush. Commonsense knowledge mining from pretrained models. In Proceedings of the 2019 Conference Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1173-1178, 2019.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. arXiv preprint 170.0:177..01476. 2017.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL), Pp. 4171- 4186, Minneapolis. Minnesota, June 2019. Association for Computational Linguistics. doi: 10. URL hhtps:: www aclweb. http.//aaattooooooool-------1
Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren. Scal- able multi-hop relational reasoning for knowledge-aware question answering. arXiv preprint urXiv:2005.00664 2020.
Joseph L. Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378, 1971.
Franceschi, M Niepert, M Pontil, and X He. Learning discrete structures for graph neural net- works. In Proceedings of ICML, volume 97. PMLR, 2019.
Ronghang Hu, Anna Rohrbach, Trevor Darrell, and Kate Saenko. Language-conditionee graph networks for relational reasoning. In Proceedings of CVPR, pp. (0294-10303. 2019.
Bo Jiang, Ziyan Zhang. Doudou Lin, Jin Tang, and Bin Luo. Semi-supervised learning with graph leaming--covoluttinnll networks. In Proceedings of CVPR, pp. 11313-11320, 2019.
10

---

Under review as a conference paper at ICLR 2021
Pavan Kapanipathi, Veronika Thost, Siva Sankalp Patel, Spencer Whitehead, Ibrahim Abdelaziz, Avinash Balakrishnan, Maria Chang, Kshitij P Fadnis, R Chulaka Gunasekara, Bassem Makni, et al. Infusing knowledge into the textual entailment task using graph convolutional networks. In Proceedings of AAAI, pp. 8074-8081, 2020.
Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. Unifiedqa: Crossing format boundaries with single qa system. arXiv preprint arXiv:2005.00700, 2020.
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. arXiv preprint orXiv:180.00668) 2018.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net-
works. In Proceedings of ICLR, 2017.
Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. KagNet: Knowledge-aware graph networks for commonsense reasoning. In Proceedings of EMNLP-IJCNLPP pp. 2829-2839, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 18653v1//v
D19-1282. URLhttps::/ www. wwwwww.www..... awt..//ww....///aatttooooo/////. 1282
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. In Proceedings of ICLR, April 2020.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: /907. / 1692, 2019.
Denis Lukovnikov, Asja Fischer, and Jens Lehmann. Pretrained transformers for simple question answering over knowledge graphs In International Semantic Web Conference, pp. 470-486. Springer, 2019.
Shangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, and Songlin Hu. Graph-based reasoning heterogeneous external knowl- edge for commonsense question answering. In Proceedings of AAAI, pp. 8449-8456, 2020.
Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari. Towards generalizable neuro-symbolic systems for commonsense question answering. In Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pp. 22-32, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/
v1/D19-6003. URL htthh:///// www www.wb..w....................... www./orgtttaooooooo heanthologe woooooogy///D 6003
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, and Yejin Choi. Exploiting struc- tural and semantic context for commonsense knowledge base completion. arXiv preprint vrXiv:191.002915. 2019.
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, and Yejin Choi. Commonsense knowledge base completion with structural and semantic context. In Proceedings of AAAI, pp- 2933-2933, 2020.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In Proceedings of EMNLP, pp- 2381-2391, 2018.
Bonan Min, Ralph Grishman, Li Wan, Chang Wang, and David Gondek. Distant supervision for relation extraction with an incomplete knowledge base. In Proceedings of NAACL-HLT, pp. 777- 782, Atlanta, Georgia, June 2013. Association for Computational Linguistics. URL https:
// //www. ac aaclwed
Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. Compositional vector space models for knowledge base completion. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 156-166 2015.

---

Under review as a conference paper at ICLR 2021
Will Norcliffe-Browe Broown, Stathis Vafeias, and Sarah Parisot. Learning conditioned graph structures for interpretable visual question answering. In Advances in neural information processing systems, pp. 2018.
Xiaoman Pan, Kai Sun, Jianshu Chen, Dian Yu, Heng Ji, Claire Cardie, and Dong Yu. Improving question answering with external knowledge. In EMNLP MRQA Workshop, pp. 27, 2019.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2463-2473, 2019.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to- text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL http:
7/jmlr wwtp:/opppppp.../////.
074. html
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski. Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. In Advances in neural information processing systems, pp. 4967-4976, 2017.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pp. 593-607. Springer, 2018a.
Michael Sejr Schlichtkrull, Thomas N. Kipf. Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Aldo Gangemi, Roberto Navigli, Mather Vidal, Pascal Hitzler, Raphaël Troncy, Laura Hollink, Anna Tor- dai, and Mehwish Alam (eds.), The Semantic Web 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings, volume 10843 of Lecture Notes in Com- puler Science, pp. 593-607, Springer, 2018b. doi: 10. 10000//977--331-99417777 https: /tts:://doi.ooo http:////ooorog/..111 httprr//1.10000/9000 9788 319 93417-4_33
URL
Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: an open multilingual graph of general knowledge. In Proceedings of AAAI, pp. 4444 4451, 2017.
Dan Su, Yan Xu, Genta Indra Winata, Peng Xu, Hyeondey Kim, Zihan Liu, and Pascale Fung. Gen- cralizing question answering system with pre-trained language model fine-tuning, In Proceedings of the 2nd Workshop Machine Reading for Question Answering, pp. 203-211, 2011..
Haitian Sun, Tania Bedrax- Weiss, and William Cohen. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. In Proceedings of EMNLP-IJCNLP, pp. 2380-2390, 2019.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA:: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of NAACL-HLT, pp. 4149 4158, Minneapolis, Minnesota, June 2019. Association for Computational L Linguistics. doi: 1.118855/v1/111-111
URL htttth:t:///// www ac www..wewbbb... http:/www.......ttoooo/oo///---.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks, arXiv preprint arXiv: 2017.
Boxin Wang. Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. Infobert: Im- proving robustness of language models from an information theoretic perspective. arXiv preprint arXiv:2010.023329, 2020a.
Peifeng Wang, Nanyun Peng, Pedro Szekely, and Xiang Ren. Connecting the dots: A knowledgeable path generator for commonsense question answering. arXiv preprint 20Xv:2205500061 2020b.
12

---

Under review as a conference paper at ICLR 2021
Xiaoyan Wang, Pavan Kapanipathi, Ryan Musa, Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz, Maria Chang, Achille Fokoue, Bassem Makni, Nicholas Mattei, and Michael Witbrock. Improv- ing natural language inference using external knowledge in the science questions domain. In Proceedings of AAAI, pp. 7208-7215. AAAI Press, 2019a. 9a. doi: 10. /tt////aii.3301133311222020 URL http::::p///........ //oii .rgg 10. 1 609 aaa v331 01 3301 7208
Xiaoyan Wang, Pavan Kapanipathi, Ryan Musa, Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz, Maria Chang. Achille Fokoue, Bassem Makni, Nicholas Mattei, et al. Improving natural language inference using external knowledge the science questions domain. Proceedings of AAAI, volume 33, pp. 7208-7215, 2019b.
Bishan Yang. Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575,22 2014.
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, and Doug Downey. G-daug: Generative data augmen- tation for commonsense reasoning. arXiv preprint wwXiv:200..115444 2020.
Donghan Yu, Ruohong Zhang, Zhengbao Jiang, Yuexin Wu, and Yiming Yang. Graph-revised convolutional network. In Proceedings of ECML-PKDD, 2019.
Yuyu Zhang, Hanjun Dai, Kamil Toraman, and Le Song. Kg 2: Learning to reason science exam questions with contextual knowledge graph embeddings. arXiv preprint arXiv:/805. /805./2393, 2018.
Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang. Xi Zhou, and Xiang Zhou. 2019.
Semantics-aware bert for language understanding. arXiv preprint
13

---

Under review as a conference paper at ICLR 2021
A IMPLEMENTATION DETAILS OF EDGE FEATURE GENERATOR
As an implementation of fgen- we adopt GPT-2 (Radford et al. 2019), which is pretrained on large wide range of tasks involving sentence generation, generator to generalize the facts from the knowledge graph. We first convert each fact (h, r, t) E F
corpora and achieves great success
into a word sequence with a "prompt-generation format: h, S, i, 8, h i, where h, F, f are the
word sequence of h, r, respectively, denotes the delimiter token used by GPT-2, and [ denotes
word sequence concatenation. We denote the synthetic sentence (k,,r,)
_ (h,rst) ..... ((...))))
(h.r.t)
and finetune GPT-2 on all synthetic sentences created from F. with the language modeling objective:
P(hur.tt
L gen
Σ
Σ
log P (o(hirs)) | 3 3..., (
(h.r:))
(l.r,))FF
After that, given any two concepts 2,), we build prompt as J0 $; új;; $] and let the model to generate the following word sequence. We denote the whole sentence (both prompt and generation) S(n,,,)) ). and the hidden states of each word during generation as h1,.... hr where T is the sentence length. We average hidden states of all words in the sentence to get the relational feature: gee ( (U), U3) = Ein h.
B DETAILS OF DATASETS
CommonsenseQA Talmor et al. 2019) is a multiple-choice QA dataset targeting commonsense. It's constructed based the knowledge in ConceptNet. Since the test set of the official split (9741/1221/1 140 for OFtrain/OFdev/ OFesst) is not publicly available, we compare our models with baseline models on the inhouse split (8500/1221/124) for IHrrain/HHdew//HHttt/* used by previous works (Lin et al. 2019 Feng et al. 2020
Wang et al. 2020b).
2018 is a multiple-choiee QA dataset modeled after openbook exams. Besides 5957 clementary-lewe science questions (4957/500/500 for train/dev/test,, it also provides an open book with 1326 core science facts. Solving the dataset requires combining facts from open book with commonsense knowledge.
OpenbookQA (Mihaylvv
CODAH (Chen et al. 2019) contains 2801 sentence completion questions testing commonsense reasoning skills. We perform 5-fold cross validation following the practice in Yang al. (2020). The authors shared their splits with us.
C CASE STUDY
We compare the graph generated by our HGN with the extracted one (GN). On the development set, there are two dominating cases and we show the representative instance of each one. Figure 5 shows the first case, where HGN prunes edges from extracted graph. Our HGN assigns the highest weights to the most helpful facts (book, Locatiinn house), (telephone book, AtLocation, house). It also downweight unhelpful fact (place, ISA, house) and invalid fact (usually, Re RelatedTo, house). Figure 6 shows the second case, where new generated facts are incorporated into reasoning. All generated facts that are kept by the model make sense in context and help identify the answer. Both cases suggest that model improve the quality of the contextualized knowledge graph compared to the current methods that only rely on extracted facts.
D COMPARED METHODS
RGCN Schlichtkrull et al. 2018a) extends Graph Convolutional Networks (GCNs) Kipf & Welling 2017) with sppccffc- transition matrices during message passing. It operates on the same graph as RN, The graph vector is calculated as g Pool((h.* v vJ).
GconAttn Waag et al. 2019b) softly aligns the nodes in question and answer and do pooling over all matching nodes to get g.
httpa: www..cco..........cco..o
14

---

Under review as a conference paper at ICLR 2021
Question:
What is a place that usually does not have an elevator and that sometimes has telephane book?
Answer: house
Triples:
(book, AtLocation. house). Edge weight: 0.48, Edge type: extracted
Graph of HGN
(telephone book, AtLocation, house), Edge weight: 0.48, Edge type: extracted
(place, IsA, house), Edge weight: 0.01, Edge type: extracted
Graph of GN
(usually, RelatedTo, house), Edge weight: 0.01, Edge type: extracted
Figure 5: Case I: Unrelated extracted facts are filtered out.
Question: Where would you find office worker gossiping with their colleagues?
Answer: water cooler
Triples:
(gossip, RelatedTo, water cooler). Edge weight: 0.09, Edge type: extracted
Graph
GN
(office, RelatedTo, cooler). Edge weight: 0.09, Edge type: extracted
(office, RelatedTo, water), Edge weight: 0.09, Edge type: extracted
(office, RelatedTo, water cooler). Edge weight: 0.09. Edge type: extracted
(office worker, AtLocation, water cooler). Edge weight: 0.02, Edge type: generated
(worker, Atlocation, water cooler), Edge weight: 0.02, Edge type: generated
Graph of HGN
(gossiping, AtLocation, water cooler). Edge weight: 0.02, Edge type: generated
Figure 6: Case II: Helpful generated facts are incorporated.
KagNet (Lin et al.) 2019) uses LSTM to encode relational paths between question and answer concepts and pool over the path embeddings for graph encoding.
HGN w/ local attention differs from HGN in that the edge weight is normalized in each node's 's neighborhood instead of the whole graph. The entropy regularization not applicable for this variant.
15


Reviews:
[{"review_id": "UdpljQPZXK", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a question answering model that is augmented with a common-sense knowledge graph (KG). The paper builds on the following two observations — (a) KGs are incomplete often lacking facts that would be needed for reasoning to answer a question. (b) Current methods over-retrieves facts (edges) from the KG leading to a lot of unrelated facts that potentially makes reasoning noisier and harder.\n\nThe paper first retrieves all possible facts from the KG connecting entities in the question and answer. However, due to the incompleteness of the KG, the retrieved subgraph might be missing important edges between entities. To deal with this, they connect all nodes between question and answer entities and initialize the embedding of the newly added edge with hidden layers of a sentence generated from a fine-tuned GPT2 language model (this important detail was mentioned in the appendix). However, currently the graph is over complete and very noisy. The proposed model then sparsifies the graph by learning edge weights via a two-step message passing process. The edge weight is learned as a function of the current edge representation and the textual representation. Lastly, an entropy term is added to the objective function to encourage more peakiness (and sparsity). \n\nThe model is tested on three common-sense QA benchmarks and on three of them they beat the baselines albeit only around 1-2%. Statistical significance of the result was not reported. Ablation study show that the efficacy of including the generated edges and pruning the graph. There was a small human-study also done where the annotators were shown a binarized graph and were asked to rate each edge. Annotators had moderate agreement between themselves in finding that the pruned graph was better than the original retrieved graph.\n\n\nStrengths:\n* Developing models that can use symbolic external knowledge present in common-sense KGs and also overcome the sparsity in KG is important and this model is a step in that direction\n* The paper achieves a little improvement in performance in all three datasets and ablation experiments are helpful in understanding the results\n* The paper is clearly written and it was easy to follow for the most part\n\nWeaknesses & clarifying questions for the authors:\n* My biggest complain of the paper in its current form is that several modeling choices were not motivated at all. For example, generating edges between nodes using GPT-2 language model is fairly non-standard. However, the paper lacks any motivation on why this is the right approach to generate facts which are not captured in a KG. What is the guarantee that GPT-2 will not hallucinate and generate a false fact and thereby adding unnecessary noise in the reasoning process.\n* Following up on the previous point, there could have been several other modeling choices. For example, instead of generating text via a language model, one could gather text (sentences) from Wikipedia or other text corpora containing the entities (which would mean the text would probably not be a false fact). These modeling choices were not explored and were not discussed. \n* The GPT-2 modeling choice was also moved to the appendix and I think it should definitely be moved to the main section of the paper as it is one of the core technical contribution of the paper.\n* Another modeling decision that was not motivated was the graph reasoning part. It is unclear to me why the edge weight is modeled as a part of the message passing process. Another (simpler) alternative could be modeling it as an edge attention, which is computed wrt the text and the current node embeddings. I would be curious to know how this simple model worked and if it didn’t why was the case.\n* Even though there are improvements across dataset, the improvements are relatively minor (<1% in few datasets). I think it would be useful to have statistical significance test.\n* Regarding the human study, if I understand correctly, was only the node and adjacent matrix shown to the annotators?. Was the relation type (KB relations and generated sentences) included too? If they were not included I think they should be because knowing the relations is also very improvement.\n* Can you elaborate on the average helpfulness score of edges in table 5? How many (what proportions) were scored 0, 1 or 2 for both the graphs? I think it would also be helpful to report how many facts all/majority of the annotators found to be helpful for both the graphs. \n\nMissing Reference: It would be nice to cite Sun et al EMNLP 2019 -- PullNet: Open Domain Question Answering with\nIterative Retrieval on Knowledge Bases and Text since one of the core contributions of that paper was to retrieve and keep only relevant facts from the KG. Relation paths in KG were explored by several works before Wang et al 2020 such as Neelakantan et al ACL 2015 - Compositional Vector Space Models for Knowledge Base Completion, Das et al EACL 2017 -- Chains of reasoning over entities, relations and text etc. It would be nice to cite those work as well.\n\nRecommendation:  In light of the current weaknesses of the paper, I am giving it a score of 5 and I look forward to the discussion.\n\n=======11/22======\n\nI am deciding to keep the same scores as before. Some of the initial concerns remain. I think the paper still lacks motivation wrt the GPT2 model generating missing edges. Thank you for getting the latest results, the paper is stronger than before and with some more work, I am confident it will be a good contribution to the research community.\n\n=====11/24======\n\nAfter having read through the explanation behind using GPT2 as edge features (and sufficient backing by 2 closely related work), I am increasing my score to 6. I think the discussion helped in somewhat convincing me that this approach would work for ConceptNet because of its limited schema.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "JzSfOkUXAkc", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a graph network (called HGN), aiming to better leverage commonsense knowledge graphs (KGs) to solve commonsense question answering and reasoning tasks, by jointly generating representations for new triples from KGs, determining relevance of the triples, and learning graph model parameters. The proposed model is tested on several tasks: CommonsenseQA, OpenbookQA, and CODAH.\n\nPros:\n-  Overall, the paper is easy to follow, although there are a number of typos or grammatical errors that need to be fixed.  The overall idea is clear.\n-  Jointly learning (pruning) the graph structure with the network parameters is interesting.\n-  The proposed model outperforms the baselines in comparison.\n-  Human evaluation is provided.\n\nCons:\n-  My major concern about this paper is the novelty and contributions in terms of methodology. Compared to existing methods (e.g., those PG models proposed (Wang et al. 2020)), the novelty of the current submission is rather limited---the proposed model of jointly generating new triples and learning (pruning) the graph structure with the network parameters is an interesting, but a pretty incremental idea. \n\n-  The empirical comparison to previous work (e.g., Wang et al. 2020) needs to be clearer to help understand the empirical advantages of the proposed models. The paper mentioned some reason of excluding PG-Full from comparison, but since PG-Global does not include static knowledge embedding and PG-full does, is the latter a more reasonable baseline to be compared with? The model does not achieve better performance than existing models on some tasks, which casts doubts on its effectiveness; e.g., whether its advantage is orthogonal to that brought by stronger models such as those performing much better on the OpenbookQA task.\n\nMore comments:\n-  The paper uses much space to discuss neural symbolic approaches. Given the vague benefit of doing so, it may be better to use the limited space to focus more on establishing the contributions w.r.t. existing models; e.g., more details about (Wang et al., 2020) can be provided and compared to in both methodological and experimental analyses.\n- The human evaluation was performed on the questions with correct questions. More analyses on the edges and weights generated for questions that were not correctly answers may help better understand the proposed model. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "55ZmquMImnm", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "=== Summary ===\n\nIn this paper, the authors propose a new approach towards incorporating knowledge graphs (KG) into commonsense QA frameworks. KGs are helpful for adding structured \"world\" information, which neural-symbolic architectures can leverage to do commonsense reasoning, e.g., \"what is the expensive resource in printing on paper?\" (paper). In such architectures, however, the authors argue that KG quality is a large impediment (e.g., missing or incorrect edges, distracting nodes, etc). Therefore, they propose a \"hybrid\" KG-based model (accordingly named \"Hybrid Graph Network\") that jointly learns to refine/augment the graph structure while also optimizing it for inference performance.\n\nExperiments are conducted on a number of commonsense reasoning tasks with multiple KG sources, and compared to relevant baselines. They also perform a user study to examine the \"helpfulness\" of the refined KGs produced by the HGN.\n\n=== Justification for Score ===\n\nThis paper is well-written and well-evaluated. The proposed method is also relatively simple and intuitively motivated. The experiments, however, only show modest (yet still positive) empirical gains. Perhaps not a game-changer for commonsense QA, but still a reasonable contribution that I would recommend for acceptance.\n\n=== Strengths ===\n\n+ The paper is clear and well-written. \n+ The experimental section is strong. The model is compared to strong baselines, and I appreciated the extra user-study on learned graph structure.\n+ The method is well-motivated, and provides (modest) empirical gains compared to some baselines.\n+ The method shows good performance with respect to increasing data efficiency (Fig. 4).\n\n=== Concerns ===\n\n- The main concern is on the empirical effectiveness of the model. The results appear to give only modest gains at best (against comparable baselines to the best of my knowledge). For a number of the results the variance is large compared to the relative difference---it would helpful to also include tests of significance for these improvements.\n\n- On OpenbookQA the model significantly underperforms T5-based models. Though I appreciate T5 is unwieldy due to its large size, it makes me question if this method indeed presents a complimentary gain, or is climbing the wrong architectural hill.\n\n=== Update After Rebuttal ===\n\nI commend the authors on a through rebuttal and active rewrites/experimentation. I still think the work is good, and can warrant acceptance. However, I still find the empirical results to be only moderate at best (though I appreciated the authors' rebuttal and significance testing). I am keeping my score the same.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an interesting step in the direction of neuro-symbolic reasoning. While there is no consensus among reviewers about the key novelty of the method, all acknowledge the interest of the direction. All of them also recognize that the submission improved greatly during the discussion phase: clarification of motivations, of experimental settings and results, of discussion with previous work.

However, despite those improvements, the submission is not yet ready for publication at ICLR. We encourage the authors to use the very detailed reviews and comments to improve the work. In particular, we encourage them to pay attention at three aspects:

1/ Comparison with large language models: the discussion wrt T5 is important. A key motivation for the proposed model is that it is bringing information and elements for QA (or other reasoning tasks) that purely scaling up language models can not bring. Or maybe they can bring the same kind of improvement but at a much lower computational cost. In any case, this is a very important point to justify the interest of such approach, and neuro-symbolic reasoning overall, empirically.

2/ Using GPT2 (or equivalent): the discussion on using GPT-2 for generating new facts is key too. It is essential to bring this description from appendix to the core of the paper. But more discussion are expected.  For instance, what if GPT-2 generates facts that are false and lead to answering and justifying a wrong answer? In other words, how does it impact the integrity of the contextualized KG? This is an essential point that needs to be worked on more thoroughly. 

3/ Overall there have been a lot of discussion to improve the motivations and the contributions. But they are not reflected in the paper necessarily. Following R2, we encourage the authors to "refocus the existing version (e.g., from vague discussion about neural-symbolic models towards establishing solid comparison to the most related previous work in various sections of the submission)"




Decision: Reject