DAGN : Discourse-Aware Graph Network for Logical Reasoning (2021)

Abstract: Recent QA with logical reasoning questions requires passage-level relations among the sentences. However, current approaches still focus on sentence-level relations interacting among tokens. In this work, we explore aggregating passage-level clues for solving logical reasoning QA by using discourse-based information. We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts. The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks. Experiments are conducted on two logical reasoning QA datasets, ReClor and LogiQA, and our proposed DAGN achieves competitive results. The source code is available at https://github.com/Eleanor-H/DAGN .

## introduction a variety of qa datasets have promoted the development of reading comprehensions, for instance, squad @xcite , hotpotqa @xcite , drop @xcite , and so on. recently, qa datasets with more complicated reasoning types, i.e., logical reasoning, are also introduced, such as reclor @xcite and logiqa @xcite . the logical questions are taken from standardized exams such as gmat and lsat, and require qa models to read complicated argument passages and identify logical relationships therein. for example, selecting a correct assumption that supports an argument, or finding out a claim that weakens an argument in a passage. such logical reasoning is beyond the capability of most of the previous qa models which focus on reasoning with entities or numerical keywords. a main challenge for the qa models is to uncover the logical structures under passages, such as identifying claims or hypotheses, or pointing out flaws in arguments. to achieve this, the qa models should first be aware of logical units, which can be sentences or clauses or other meaningful text spans, then identify the logical relationships between the units. however, the logical structures are usually hidden and difficult to be extracted, and most datasets do not provide such logical structure annotations. an intuitive idea for unwrapping such logical information is using discourse relations. for instance, as a conjunction, "because" indicates a causal relationship, whereas "if" indicates a hypothetical relationship. however, such discourse-based information is seldom considered in logical reasoning tasks. modeling logical structures is still lacking in logical reasoning tasks, while current opened methods use contextual pre-trained models @xcite . besides, previous graph-based methods @xcite ) that construct entity-based graphs are not suitable for logical reasoning tasks because of different reasoning units. in this paper, we propose a new approach to solve logical reasoning qa tasks by incorporating discourse-based information. first, we construct discourse structures. we use discourse relations from the penn discourse treebank 2.0 (pdtb 2.0) @xcite as delimiters to split texts into elementary discourse units (edus). a logic graph is constructed in which edus are nodes and discourse relations are edges. then, we propose a discourse-aware graph network (dagn) for learning high-level discourse features to represent passages.the discourse features are incorporated with the contextual token features from pre-trained language models. with the enhanced features, dagn predicts answers to logical questions. our experiments show that dagn surpasses current opened methods on two recent logical rea-large-scale pre-trained model a signal…detailed [while] digital system…units [.] with this …disadvantage. [since] there is…signal [,] the duplication… original [,] which are errors [.] context edus e 1 e 2 e 3 e 4 e 5 e 1 e 2 e 3 e 4 e 5 e 6 e 7 input @xmath0.] answer option edus e 7 e 6 e 1 e 7 e 2 e 3 e 4 e 5 e 8 updated edu @xmath1 theoretically, analog systems are superior to digital systems. a signal in a pure analog system can be infinitely detailed, while digital systems cannot produce signals that are more precise than their digital units. with this theoretical advantage there is a practical disadvantage. since there is no limit on the potential detail of the signal, the duplication of an analog representation allows tiny variations from the original, which are errors. context: question: the statements above, if true, most strongly support which one of the following? digital systems are the best information systems because error cannot occur in the emission of digital signals. figure 1: the architecture of our proposed method with an example below. ## method our intuition is to explicitly use discourse-based information to mimic the human reasoning process for logical reasoning questions. the questions are in multiple choices format, which means given a triplet (context, question, answer options), models answer the question by selecting the correct answer option. our framework is shown in figure 1 . we first construct a discourse-based logic graph from the raw text. then we conduct reasoning via graph networks to learn and update the discoursebased features, which are incorporated with the contextual token embeddings for downstream answer prediction. ## experiments we evaluate the performance of dagn on two logical reasoning datasets, reclor @xcite and logiqa @xcite , and conduct ablation study on graph construction and graph network. the implementation details are shown in appendix b. ## related works recent datasets for reading comprehension tend to be more complicated and require models' capability of reasoning. for instance, hotpotqa @xcite , wikihop @xcite , open-bookqa @xcite , and multirc @xcite require the models to have multi-hop reasoning. drop @xcite and ma-taco @xcite need the models to have numerical reasoning. wiqa @xcite and cosmosqa @xcite require causal reasoning that the models can understand the counterfactual hypothesis or find out the causeeffect relationships in events. however, the logical reasoning datasets @xcite require the models to have the logical reasoning capability of uncovering the inner logic of texts. deep neural networks are used for reasoningdriven rc. evidence-based methods @xcite @xcite generate explainable evidence from a given context as the backup of reasoning. graph-based methods @xcite @xcite @xcite @xcite explicitly model the reasoning process with constructed graphs, then learn and update features through message passing based on graphs. there are also other methods such as neuro-symbolic models @xcite and adversarial training @xcite . our paper uses a graph-based model. however, for uncovering logical relations, graph nodes and edges are customized with discourse information. discourse information provides a high-level understanding of texts and hence is beneficial for many of the natural language tasks, for instance, text summarization @xcite @xcite , neural machine translation @xcite , and coherent text generation @xcite . there are also discourse-based applications for reading comprehension. discern @xcite segments texts into edus and learns interactive edu features. @xcite provide additional discourse-based annotations and encodes them with discourseaware self-attention models. unlike previous works, dagn first uses discourse relations as graph edges connecting edus for texts, then learns the discourse features via message passing with graph neural networks. ## conclusion in this paper, we introduce a discourse-aware graph network (dagn) to addressing logical reasoning qa tasks. we first treat elementary discourse units (edus) that are split by discourse relations as basic reasoning units. we then build discourse-based logic graphs with edus as nodes and discourse relations as edges. dagn then learns the discourse-based features and enhances them with contextual token embeddings. dagn reaches competitive performances on two recent logical reasoning datasets reclor and logiqa.