  Conventional Machine Reading Comprehension (MRC) has been well-addressed by
pattern matching, but the ability of commonsense reasoning remains a gap
between humans and machines. Previous methods tackle this problem by enriching
word representations via pre-trained Knowledge Graph Embeddings (KGE). However,
they make limited use of a large number of connections between nodes in
Knowledge Graphs (KG), which could be pivotal cues to build the commonsense
reasoning chains. In this paper, we propose a Plug-and-play module to
IncorporatE Connection information for commonsEnse Reasoning (PIECER). Beyond
enriching word representations with knowledge embeddings, PIECER constructs a
joint query-passage graph to explicitly guide commonsense reasoning by the
knowledge-oriented connections between words. Further, PIECER has high
generalizability since it can be plugged into suitable positions in any MRC
model. Experimental results on ReCoRD, a large-scale public MRC dataset
requiring commonsense reasoning, show that PIECER introduces stable performance
improvements for four representative base MRC models, especially in
low-resource settings.
