Automated Graph Generation at Sentence Level for Reading Comprehension Based on Conceptual Graphs (2020)

Abstract: This paper proposes a novel miscellaneous-context-based method to convert a sentence into a knowledge embedding in the form of a directed graph. We adopt the idea of conceptual graphs to frame for the miscellaneous textual information into conceptual compactness. We first empirically observe that this graph representation method can (1) accommodate the slot-filling challenges in typical question answering and (2) access to the sentence-level graph structure in order to explicitly capture the neighbouring connections of reference concept nodes. Secondly, we propose a task-agnostic semantics-measured module, which cooperates with the graph representation method, in order to (3) project an edge of a sentence-level graph to the space of semantic relevance with respect to the corresponding concept nodes. As a result of experiments on the QA-type relation extraction, the combination of the graph representation and the semantics-measured module achieves the high accuracy of answer prediction and offers human-comprehensible graphical interpretation for every well-formed sample. To our knowledge, our approach is the first towards the interpretable process of learning vocabulary representations with the experimental evidence.

## introduction in the ideal world, a knowledge representation technique, which solves the gamut of reading comprehension tasks from question answering to university entrance examinations, shows the ubiquity and understanding in all the corpora. in fact, for machine reading comprehension, developing a ubiquitous and well-defined knowledge representation which can be applied to open domain is still thorny in practice. from the aspects of linguistics and neurophysiology, the study of syntax-semantics interface @xcite explains the effect of syntactic properties on semantic interpretation. that is why the syntaxsemantics interface is of paramount importance for reading comprehension. in particular, the extraction of syntactic relations between the verb and its arguments plays a crucial role in the comprehension of language @xcite . similarly, the abstract hierarchical syntactic structure of language is considered imperative for on-line comprehension even in a task-free environment @xcite . as one of methods which integrate the syntactic-semantic information, our graph generation model gains a better insight into the relationship between language processing and graph theory. in general, graph representation approaches regarding the syntax-semantics interface, are more suitable for the analyses of the relations between open-domain reading comprehension and hierarchical structure of syntax. in our case, we have the advantage in processing textual data effectively due to the combination of the theories of syntax-semantics interface and the practice of graph generation. by making use of hierarchical syntactic structure, our context-based graph model is beneficial to find the interactions and meanings between words; particularly, it is the fact that graphs capture interactions more efficiently between individual units of contextual data @xcite . drawing our inspiration from conceptual graphs @xcite , we propose a miscellaneous-contextbased method which allows a single sentence to be converted into a directed graph based on them, due to these logic-designed knowledge representations appropriate for semantic interpretation, higher-order expressions and metalevel operations of natural languages. improving the reading comprehensibility, our mechanism generates a semantic network comprising the concept entities as graphical units for each sentence, and thence a piece of integrated information about each extracted concept entity is independently accessible to computer systems and comprehensible to human beings. utilising the hypernym hierarchies provided by wordnet @xcite , we therefore project a sentence-level network to a large-scale lexical ontology for more explicit semantic positioning of concept entities, due to the fact that wordnet is designed by psycholinguistic features as well. to our knowledge, our approach is the first towards the interpretable process of learning vocabulary representations with the experimental evidence. here comes the emphases upon our contributions: ## related work in the early stage of machine reading comprehension, a system @xcite was first developed in usage of pattern matching techniques augmented with linguistic processing, for questionanswering tasks. following the trend of machine learning, a large-scale training dataset and a deep neural network cooperate on confronting the challenges of reading documents with less prior knowledge of language structure @xcite . following in the context of flourishing machine learning techniques, the drqa model @xcite and the reinforced ranker-reader system @xcite demonstrate the machine reading at scale through multitask learning on a variety of question-answering tasks. @xcite without relying on any task-specific modules or parameters, demonstrates the improvement in transfer learning for a wide diversity of nlp tasks, such as machine translation, natural language inference and semantic parsing, etc. graphs are beneficial for the access to the structure of human knowledge. a graph database, known as freebase @xcite , has relatively more definite data of general knowledge about types and properties, despite the limited capacity for knowledge expansion. as for question-answering tasks, the gtcr @xcite and kg-mrc systems @xcite both predict the answers, involving temporal and causal relations, through the generation of specific knowledge graphs. for multi-hop reading comprehension, which requires multiple pieces of evidence to answer a question correctly, mhqa @xcite introduces the graph convolutional network (gcn) and graph recurrent network (grn) to integrate and connect global evidence better. recently, graph representations gradually appear to address a wide range of reading comprehension challenges. for example, gtcr @xcite generates event graphs for answering the temporal and causal questions, kg-mrc @xcite builds dynamic knowledge graphs for tracking the evolving states of participant entities, and the other variations of knowledge graph embeddings @xcite @xcite @xcite @xcite have been presented. ## proposed methods in view of applications to the open domain of corpora, we bind a knowledge representation to generalisation of arbitrary well-formed text (i.e., the text conforms the grammatical well-formedness developed in generative linguistics.), for the purpose of dealing with reading comprehension issues. to this end, we propose a new method to automatically convert a sentence into a directed graph based on conceptual graphs, which provides the comprehensible format for not only computer systems but also human beings. unlike most methods in exploiting machine learning techniques for feature extraction, this graphical knowledge embedding model consists of feature capture, synthesis, inference, tree-based structure forming, and semantic role labelling. ## experimental results and comparison in this section, we evaluate the pipeline that attaches the graph representation agg-cg to the module smm on the question-answering-based task of relation extraction. according to the slot-filling data collected from wikipedia, it allows us to demonstrate the ability of this graph representation method for narrowing down the candidates for the answer, through the access of graphs' hierarchical trees at a certain level. ## analysis in this section, we demonstrate how the parameter setup of smm impacts the performance of the whole pipeline and how the conceptual graph representation acts as a critical role of different pipelines in natural language processing tasks. ## conclusion and future work the main theme of this paper presents the graph method for knowledge representations to process textual data at sentence level, which is designed to tackle open-domain corpora but not serve as a task-oriented model for reading comprehension tasks. in future work, we will consider enhancing semantic role labelling between nodes and conceive an algorithm of graph composition between sentences at paragraph level, in order to combine the language with more complicated geometric analysis and graph theory. from the experimental aspect, we will demonstrate that our graph representation cooperating with a simple task-oriented module can be evaluated on more qa-type tasks, for presenting the ubiquity of agg-cg representations on natural language understanding.