Venue: NAACL 2021

# DAGN: Discourse-Aware Graph Network for Logical Reasoning

Yinya Huang\({}^{1}\) Meng Fang\({}^{2}\) Yu Cao\({}^{3}\) Liwei Wang\({}^{4}\) Xiaodan Liang\({}^{1}\)

\({}^{1}\)Shenzhen Campus of Sun Yat-sen University

\({}^{2}\)Tencent Robotics X

\({}^{3}\)School of Computer Science, The University of Sydney

\({}^{4}\)The Chinese University of Hong Kong

yinga.huang@hotmail, mfang@tencent.com, ycao8647@uni.sydney.edu.au, lwwang@cse.cuhk.edu.hk, xdliang328@gmail.com

This work was done during Yinya Huang's internship in Tencent with L. Wang and M. Fang.Corresponding Author: Xiaodan Liang.

###### Abstract

Recent QA with logical reasoning questions requires passage-level relations among the sentences. However, current approaches still focus on sentence-level relations interacting among tokens. In this work, we explore aggregating passage-level clues for solving logical reasoning (QA by using discourse-based information. We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts. The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks. Experiments are conducted on two logical reasoning QA datasets, RecIcor and LogiQA, and our proposed DAGN achieves competitive results. The source code is available at [https://github.com/Eleanor-H/DAGN](https://github.com/Eleanor-H/DAGN).

## 1 Introduction

A variety of QA datasets have promoted the development of reading comprehensions, for instance, SQuAD Rajpurkar et al. (2016), HotpotQA Yang et al. (2018), DROP Dua et al. (2019), and so on. Recently, QA datasets with more complicated reasoning types, i.e., logical reasoning, are also introduced, such as ReClor Yu et al. (2020) and LogiQA Liu et al. (2020). The logical questions are taken from standardized exams such as GMAT and LSAT, and require QA models to read complicated argument passages and identify logical relationships therein. For example, selecting a correct assumption that supports an argument, or finding out a claim that weakens an argument in a passage. Such logical reasoning is beyond the capability of most of the previous QA models which focus on reasoning with entities or numerical keywords.

A main challenge for the QA models is to uncover the logical structures under passages, such as identifying claims or hypotheses, or pointing out flaws in arguments. To achieve this, the QA models should first be aware of logical units, which can be sentences or clauses or other meaningful text spans, then identify the logical relationships between the units. However, the logical structures are usually hidden and difficult to be extracted, and most datasets do not provide such logical structure annotations.

An intuitive idea for unwrapping such logical information is using discourse relations. For instance, as a conjunction, "because" indicates a causal relationship, whereas "if" indicates a hypothetical relationship. However, such discourse-based information is seldom considered in logical reasoning tasks. Modeling logical structures is still lacking in logical reasoning tasks, while current opened methods use contextual pre-trained models Yu et al. (2020). Besides, previous graph-based methods Ran et al. (2019); Chen et al. (2020) that construct entity-based graphs are not suitable for logical reasoning tasks because of different reasoning units.

In this paper, we propose a new approach to solve logical reasoning QA tasks by incorporating discourse-based information. First, we construct discourse structures. We use discourse relations from the Penn Discourse TreeBank 2.0 (PDTB 2.0) Prasad et al. (2008) as delimiters to split texts into elementary discourse units (EDUs). A logic graph is constructed in which EDUs are nodes and discourse relations are edges. Then, we propose a Discourse-Aware Graph Network (DAGN) for learning high-level discourse features to represent passages.The discourse features are incorporated with the contextual token features from pre-trained language models. With the enhanced features, DAGN predicts answers to logical questions. Our experiments show that DAGN surpasses current opened methods on two recent logical re

sioning QA datasets, ReClor and LogiQA.

Our main contributions are three-fold:

* We propose to construct logic graphs from texts by using discourse relations as edges and elementary discourse units as nodes.
* We obtain discourse features via graph neural networks to facilitate logical reasoning in QA models.
* We show the effectiveness of using logic graph and feature enhancement by noticeable improvements on two datasets, ReClor and LogiQA.

## 2 Method

Our intuition is to explicitly use discourse-based information to mimic the human reasoning process for logical reasoning questions. The questions are in multiple choices format, which means given a triplet (context, question, answer options), models answer the question by selecting the correct answer option. Our framework is shown in Figure 1. We first construct a discourse-based logic graph from the raw text. Then we conduct reasoning via graph networks to learn and update the discourse-based features, which are incorporated with the contextual token embeddings for downstream answer prediction.

### Graph Construction

Our discourse-based logic graph is constructed via two steps: delimiting text into elementary discourse units (EDUs) and forming the graph using their relations as edges, as illustrated in Figure 1(1).

Discourse Units DelimitationIt is studied that clause-like text spans delimited by discourse relations can be discourse units that reveal the rhetorical structure of texts Mann and Thompson (1988); Prasad et al. (2008). We further observe that such discourse units are essential units in logical reasoning, such as being assumptions or opinions. As the example shown in Figure 1, the "_while_" in the context indicates a comparison between the attributes of "_pure analog system_" and that of "_digital systems_". The "_because_" in the option provides evidence "_error cannot occur in the emission of digital signals_" to the claim "_digital systems are the best information systems_".

We use PDTB 2.0 Prasad et al. (2008) to help drawing discourse relations. PDTB 2.0 contains discourse relations that are manually annotated on the 1 million Wall Street Journal (WSJ) corpus and are broadly characterized into "Explicit" and "Implicit" connectives. The former apparently presents in sentences such as discourse adverbial "_instead_" or subordinating conjunction "_because_", whereas the latter are inferred by annotators between successive pairs of text spans split by punctuation marks

Figure 1: The architecture of our proposed method with an example below.

 

such as "." or ";". We simply take all the "Explicit" connectives as well as common punctuation marks to form our discourse delimiter library (details are given in Appendix A), with which we delimit the texts into EDUs. For each data sample, we segment the context and options, ignoring the question since the question usually does not carry logical content.

Discourse Graph ConstructionWe define the discourse-based graphs with EDUs as nodes, the "Explicit" connectives as well as the punctuation marks as two types of edges. We assume that each connective or punctuation mark connects the EDUs before and after it. For example, the option sentence in Figure 1 is delimited into two EDUs, \({\rm EDU}_{7}=\)"_digital systems are the best information systems_" and \({\rm EDU}_{8}=\)"_error cannot occur in the emission of digital signals_" by the connective \(r=\)"_because_". Then the returned triplets are \(({\rm EDU}_{7},r,{\rm EDU}_{8})\) and \(({\rm EDU}_{8},r,{\rm EDU}_{7})\). For each data sample with the context and multiple answer options, we separately construct graphs corresponding to each option, with EDUs in the same context and every single option. The graph for the single option \(k\) is denoted by \(\mathcal{G}^{k}=(\mathcal{V}^{k},\mathcal{E}^{k})\).

### Discourse-Aware Graph Network

We present the Discourse-Aware Graph Network (DAGN) that uses the constructed graph to exploit discourse-based information for answering logical questions. It consists of three main components: an EDU encoding module, a graph reasoning module, and an answer prediction module. The former two are demonstrated in Figure 1(2), whereas the final component is in Figure 1(3).

EDU EncodingAn EDU span embedding is obtained from its token embeddings. There are two steps. First, similar to previous works Yu et al. (2020); Liu et al. (2020), we encode such input sequence "
 context 
 question ||option 
" into contextual token embeddings with pre-trained language models, where 
 and 
 are the special tokens for RoBERTa Liu et al. (2019) model, and | | denotes concatenation. Second, given the token embedding sequence \(\{{\bf t}_{1},{\bf t}_{2},...,{\bf t}_{L}\}\), the \(n\)-th EDU embedding is obtained by \({\bf e}_{n}=\sum\limits_{l\in S_{n}}{\bf t}_{l}\), where \(S_{n}\) is the set of token indices belonging to \(n\)-th EDU.

Graph ReasoningAfter EDU encoding, DAGN performs reasoning over the discourse graph. Inspired by previous graph-based models Ran et al. (2019); Chen et al. (2020), we also learn graph node representations to obtain higher-level features. However, we consider different graph construction and encoding. Specifically, let \(\mathcal{G}^{k}=(\mathcal{V}^{k},\mathcal{E}^{k})\) denote a graph corresponding to the \(k\)-th option in answer choices. For each node \(v_{i}\in{\bf V}\), the node embedding \({\bf v}_{i}\) is initialized with the corresponding EDU embedding \({\bf e}_{i}\). \(\mathcal{N}_{i}=\{j|(v_{j},v_{i})\in\mathcal{E}^{k}\}\) indicates the neighbors of node \(v_{i}\). \({\bf W}^{r_{ji}}\) is the adjacency matrix for one of the two edge types, where \(r_{E}\) indicates graph edges corresponding to the explicit connectives, and \(r_{I}\) indicates graph edges corresponding to punctuation marks.

The model first calculates weight \(\alpha_{i}\) for each node with a linear transformation and a sigmoid function \(\alpha_{i}=\sigma({\bf W}^{\alpha}({\bf v}_{i})+b^{\alpha})\), then conducts message propagation with the weights:

\[\tilde{\mathbf{v}}_{i}=\frac{1}{|\mathcal{N}_{i}|}(\sum_{j\in\mathcal{N}_{i}} \alpha_{j}{\bf W}^{r_{ji}}{\bf v}_{j}),r_{ji}\in\{r_{E},r_{I}\} \tag{1}\]

where \(\tilde{\mathbf{v}}_{i}\) is the message representation of node \(v_{i}\). \(\alpha_{j}\) and \({\bf v}_{j}\) are the weight and the node embedding of \(v_{j}\) respectively.

After the message propagation, the node representations are updated with the initial node embeddings and the message representations by

\[{\bf v}^{\prime}_{i}={\text{ReLU}}({\bf W}^{u}{\bf v}_{i}+\tilde{\mathbf{v}}_ {i}+{\bf b}^{u}), \tag{2}\]

where \({\bf W}^{u}\) and \({\bf b}^{u}\) are weight and bias respectively. The updated node representations \({\bf v}^{\prime}_{i}\) will be used to enhance the contextual token embedding via summation in corresponding positions. Thus \({\bf t}^{\prime}_{l}={\bf t}_{l}+{\bf v}^{\prime}_{n}\), where \(l\in S_{n}\) and \(S_{n}\) is the corresponding token indices set for \(n\)-th EDU.

Answer PredictionThe probabilities of options are obtained by feeding the discourse-enhanced token embeddings into the answer prediction module. The model is end-to-end trained using cross entropy loss. Specifically, the embedding sequence first goes through a layer normalization Ba et al. (2016), then a bidirectional GRU Cho et al. (2014). The output embeddings are then added to the input ones as the residual structure He et al. (2016). We finally obtain the encoded sequence after another layer normalization on the added embeddings.

We then merge the high-level discourse features and the low-level token features. Specifically, the variant-length encoded context sequence, questionnaire-option sequence are pooled via weighted summation wherein the weights are softmax results of 

a linear transformation of the sequence, resulting in single feature vectors separately. We concatenate them with "\(<\)s\(>\)" embedding from the backbone pre-trained model, and feed the new vector into a two-layer perceptron with a GELU activation Hendrycks and Gimpel (2016) to get the output features for classification.

## 3 Experiments

We evaluate the performance of DAGN on two logical reasoning datasets, ReClor Yu et al. (2020) and LogiQA Liu et al. (2020), and conduct ablation study on graph construction and graph network. The implementation details are shown in Appendix B.

### Datasets

ReClor contains 6,138 questions modified from standardized tests such as GMAT and LSAT, which are split into train / dev / test sets with 4,638 / 500 / 1,000 samples respectively. The training set and the development set are available. The test set is blind and hold-out, and split into an EASY subset and a HARD subset according to the performance of BERT-base model Devlin et al. (2019). The test results are obtained by submitting the test predictions to the leaderboard. LogiQA consists of 8,678 questions that are collected from National Civil Servants Examinations of China and manually translated into English by professionals. The dataset is randomly split into train / dev / test sets with 7,376 / 651 / 651 samples respectively. Both datasets contain multiple logical reasoning types.

### Results

The experimental results are shown in Tables 1 and 2. Since there is no public method for both datasets, we compare DAGN with the baseline models. As for DAGN, we fine-tune RoBERTA-Large as the backbone. DAGN (Aug) is a variant that augments the graph features.

DAGN reaches 58.20% of test accuracy on ReClor. DAGN (Aug) reaches 58.30%, therein 75.91% on EASY subset, and 44.46% on HARD subset. Compared with RoBERTa-Large, the improvement on the HARD subset is remarkably 4.46%. This indicates that the incorporated discourse-based information supplements the shortcoming of the baseline model, and that the discourse features are beneficial for such logical reasoning. Besides, DAGN and DAGN (Aug) also outperform the baseline models on LogiQA, especially showing 4.01% improvement over RoBERTa-Large on the test set.

### Ablation Study

We conduct ablation study on graph construction details as well as the graph reasoning module. The results are reported in Table 3.

Varied Graph NodesWe first use clauses or sentences in substitution for EDUs as graph nodes. For clause nodes, we simply remove "Explicit" connectives during discourse unit delimitation. So that the texts are just delimited by punctuation marks. For sentence nodes, we further reduce the delimiter library to solely period ("."). Using the modified graphs with clause nodes or coarser sentence nodes, the accuracy of DAGN drops to 64.40%. This indicates that clause or sentence nodes carry

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Methods** & **Dev** & **Test** \\ \hline BERT-Large & 53.80 & 49.80 & 72.00 & 32.30 \\ XLNet-Large & 62.00 & 56.00 & 75.70 & 40.50 \\ RoBERTa-Large & 62.60 & 55.60 & 75.50 & 40.00 \\ DAGN & **65.20** & **58.20** & **76.14** & **44.11** \\ DAGN (Aug) & **65.80** & **58.30** & **75.91** & **44.46** \\ \hline \hline \end{tabular}

* The results are taken from the ReClor paper.
* DAGN ranks the 1st on the public ReClor leaderboard’until 17th Nov., 2020 before submitting it to NAACL. Until now, we find that several better results appeared in the leaderboard and they are not opened.

\end{table}
Table 1: Experimental results (accuracy %) of DAGN compared with baseline models on ReClor dataset. Test-E = Test-EASY, Test-H = Test-HARD.

\begin{table}
\begin{tabular}{l r} \hline \hline
* clause nodes & 64.40 \\ DAGN
- sentence nodes & 64.40 \\ _ablation on edges_ & \\ DAGN
- single edge type & 64.80 \\ DAGN
- fully connected edges & 61.60 \\ _ablation on graph reasoning_ & DAGN w/o graph module & 64.00 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experimental results (accuracy %) of DAGN compared with baseline models on LogiQA dataset.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Methods** & **Dev** \\ \hline BERT-Large & 34.10 & 31.03 \\ RoBERTa-Large & 35.02 & 35.33 \\ DAGN & **35.48** & **38.71** \\ DAGN (Aug) & **36.87** & **39.32** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation study results (accuracy %) on ReClor development set.

 

less discourse information and act poorly as logical reasoning units.

Varied Graph EdgesWe make two changes of the edges: (1) modifying the edge type, (2) modifying the edge linking. For edge type, all edges are regarded as a single type. For edge linking, we ignore discourse relations and connect every pair of nodes, turning the graph into fully-connected. The resulting accuracies drop to 64.80% and 61.60% respectively. It is proved that in the graph we built, edges link EDUs in reasonable manners, which properly indicates the logical relations.

Ablation on Graph ReasoningWe remove the graph module from DAGN and give a comparison. This model solely contains an extra prediction module than the baseline. The performance on ReClor dev set is between the baseline model and DAGN. Therefore, despite the prediction module benefits the accuracy, the lack of graph reasoning leads to the absence of discourse features and degenerates the performance. It demonstrates the necessity of discourse-based structure in logical reasoning.

## 4 Related Works

Recent datasets for reading comprehension tend to be more complicated and require models' capability of reasoning. For instance, HotpotQA Yang et al. (2018), WikiHop Welbl et al. (2018), OpenBookQA Mihaylov et al. (2018), and MultiRC Khashabi et al. (2018) require the models to have multi-hop reasoning. DROP Dua et al. (2019) and MA-TACO Zhou et al. (2019) need the models to have numerical reasoning. WIQA Tandon et al. (2019) and CosmosQA Huang et al. (2019) require causal reasoning that the models can understand the counterfactual hypothesis or find out the cause-effect relationships in events. However, the logical reasoning datasets Yu et al. (2020); Liu et al. (2020) require the models to have the logical reasoning capability of uncovering the inner logic of texts.

Deep neural networks are used for reasoning-driven RC. Evidence-based methods Madaan et al. (2020); Huang et al. (2020); Rajagopal et al. (2020) generate explainable evidence from a given context as the backup of reasoning. Graph-based methods Qiu et al. (2019); De Cao et al. (2019); Cao et al. (2019); Ran et al. (2019); Chen et al. (2020b); Xu et al. (2020) explicitly model the reasoning process with constructed graphs, then learn and update features through message passing based on graphs. There are also other methods such as neuro-symbolic models Saha et al. (2021) and adversarial training Pereira et al. (2020). Our paper uses a graph-based model. However, for uncovering logical relations, graph nodes and edges are customized with discourse information.

Discourse information provides a high-level understanding of texts and hence is beneficial for many of the natural language tasks, for instance, text summarization Cohan et al. (2018); Joty et al. (2019); Xu et al. (2020); Feng et al. (2020), neural machine translation Voita et al. (2018), and coherent text generation Wang et al. (2020); Bosselut et al. (2018). There are also discourse-based applications for reading comprehension. DISCERN Gao et al. (2020) segments texts into EDUs and learns interactive EDU features. Mihaylov and Frank (2019) provide additional discourse-based annotations and encodes them with discourse-aware self-attention models. Unlike previous works, DAGN first uses discourse relations as graph edges connecting EDUs for texts, then learns the discourse features via message passing with graph neural networks.

## 5 Conclusion

In this paper, we introduce a Discourse-Aware Graph Network (DAGN) to addressing logical reasoning QA tasks. We first treat elementary discourse units (EDUs) that are split by discourse relations as basic reasoning units. We then build discourse-based logic graphs with EDUs as nodes and discourse relations as edges. DAGN then learns the discourse-based features and enhances them with contextual token embeddings. DAGN reaches competitive performances on two recent logical reasoning datasets ReClor and LogiQA.

### Acknowledgements

The authors would like to thank Wenge Liu, Jianheng Tang, Guanlin Li and Wei Wang for their support and useful discussions. This work was supported in part by National Natural Science Foundation of China (NSFC) under Grant No.U19A2073 and No.61976233, Guangdong Province Basic and Applied Basic Research (Regional Joint Fund-Key) Grant No.2019B1515120039, Shenzhen Basic Research Project (Project No. JCYJ20190807154211365), Zhijiang Lab's Open Fund (No. 2020AA3AB14) and CSIG Young Fellow Support Fund 



## References

* [1]J. Lei Ba, J. R. Kiros, and G. E. Hinton (2016) Layer normalization. stat1050, pp. 21. Cited by: SS3.4.
* A. Bosselut, A. Celikyilmaz, X. He, J. Gao, P. Huang, and Y. Choi (2018) Discourse-aware neural rewards for coherent text generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Vol. 173-184, New Orleans, Louisiana, pp. 173-184. External Links: Link Cited by: SS1.
* Y. Cao, M. Fang, and D. Tao (2019) Bag: bi-directional attention entity graph convolutional network for multi-hop reasoning question answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 357-362. Cited by: SS1.
* K. Chen, W. Xu, X. Cheng, Z. Xiaochuan, Y. Zhang, L. Song, T. Wang, Y. Qi, and Wei Chu (2020a) Question directed graph attention network for numerical reasoning over text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 6759-6768. Cited by: SS1.
* K. Cheu, W. Xu, X. Cheng, Z. Xiaochuan, Y. Zhang, L. Song, T. Wang, Y. Qi, and W. Chu (2020b) Question directed graph attention network for numerical reasoning over text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 6759-6768. Cited by: SS1.
* K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio (2014) On the properties of neural machine translation: encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103-111. Cited by: SS1.
* A. Cohan, F. Dernoncourt, D. Soon Kim, T. Bui, S. Kim, W. Chang, and N. Golharian (2018) A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana, pp. 615-621. Cited by: SS1.
* N. De Cao, W. Aziz, and I. Titov (2019) Question answering by reasoning across documents with graph convolutional networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2306-2317. Cited by: SS1.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186. Cited by: SS1.
* D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner (2019) Drop: a reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2368-2378. Cited by: SS1.
* X. Feng, X. Feng, B. Qin, X. Geng, and T. Liu (2020) Dialogue discourse-aware graph convolutional networks for abstractive meeting summarization. arXiv preprint arXiv:2012.03502. Cited by: SS1.
* Y. Gao, C. Wu, J. Li, S. Joty, S. CH Hoi, C. Xiong, Irwin King, and M. Lyu (2020) Discern: discourse-aware entailment reasoning network for conversational machine reading. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2439-2449. Cited by: SS1.
* K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. Cited by: SS1.
* D. Hendrycks and K. Gimpel (2016) Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. Cited by: SS1.
* L. Huang, R. Le Bras, C. Bhagavatula, and Y. Choi (2019) Cosmos qa: machine reading comprehension with contextual commonsense reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2391-2401. Cited by: SS1.
* Y. Huang, M. Fang, X. Zhan, Q. Cao, X. Liang, and L. Lin (2020) Remember: recursive erasure memory network for commonsense evidence refinement. arXiv preprint arXiv:2012.13185. Cited by: SS1.
* S. Joty, G. Careinni, R. Ng, and G. Murray (2019) Discourse analysis and its applications. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, Florence, Italy, pp. 12-17. Cited by: SS1.



Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 252-262.
* Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _ICLR (Poster)_.
* Liu et al. (2020) Jian Liu, Leyang Cui, Hanneming Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. _IJCAI 2020_.
* Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Dani Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Madaan et al. (2020) Aman Madaan, Dheeraj Rajagopal, Yiming Yang, Abhilasha Ravichander, Eduard Hovy, and Shirmai Prabhumoye. 2020. Eigen: Event influence generation using pre-trained language models. _arXiv preprint arXiv:2010.11764_.
* Mann and Thompson (1988) William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. _Text_, 8(3):243-281.
* Mihalyov et al. (2018) Todor Mihalyov, Peter Clark, Tshar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 2381-2391.
* Mihalyov and Frank (2019) Todor Mihalyov and Anette Frank. 2019. Discourse-aware semantic self-attention for narrative reading comprehension. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2541-2552, Hong Kong, China. Association for Computational Linguistics.
* Pereira et al. (2020) Lis Pereira, Xiaodong Liu, Fei Cheng, Masayuki Asahara, and Ichiro Kobayashi. 2020. Adversarial training for commonsense inference. In _Proceedings of the 5th Workshop on Representation Learning for NLP_, pages 55-60, Online. Association for Computational Linguistics.
* Prasad et al. (2008) Rashmi Prasad, Nikihil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008. The penn discourse treebank 2.0. In _LREC_. Citeseer.
* Qiu et al. (2019) Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019. Dynamically fused graph network for multi-hop reasoning. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 6140-6150, Florence, Italy. Association for Computational Linguistics.
* Rajagopal et al. (2020) Dheeraj Rajagopal, Nister Tandon, Peter Clark, Bhavana Dalvi, and Eduard Hovy. 2020. What-if i ask you to explain: Explaining the effects of perturbations in procedural text. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings_, pages 3345-3355.
* Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 2383-2392.
* Ran et al. (2019) Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. 2019. Numnet: Machine reading comprehension with numerical reasoning. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2474-2484.
* Saha et al. (2021) Amrita Saha, Shafiq Joty, and Steven CH Hoi. 2021. Weakly supervised neuro-symbolic module networks for numerical reasoning. _arXiv preprint arXiv:2101.11802_.
* Tandon et al. (2019) Niket Tandon, Binvana Dalvi, Keisuke Sakaguchi, Peter Clark, and Antoine Bosseltu. 2019. Wiga: A dataset for "what if..." reasoning over procedural text. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 6078-6087.
* Voita et al. (2018) Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. Context-aware neural machine translation learns anaphora resolution. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1264-1274.
* Wang et al. (2020) Wei Wang, Piji Li, and Hai-Tao Zheng. 2020. Consistency and coherency enhanced story generation. _arXiv preprint arXiv:2010.08822_.
* Welbl et al. (2018) Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. Constructing datasets for multi-hop reading comprehension across documents. _Transactions of the Association for Computational Linguistics_, 6:287-302.
* Xu et al. (2020a) Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020a. Discourse-aware neural extractive text summarization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 5021-5031, Online. Association for Computational Linguistics.
* Xu et al. (2020b) Yunqiu Xu, Meng Fang, Ling Chen, Yali Du, Joey Tianyi Zhou, and Chengqi Zhang. 2020b. Deep 

reinforcement learning with stacked hierarchical attention for text-based games. In Advances in Neural Information Processing Systems, Vol. 33, pp. 16495-16507. Curran Associates, Inc. Cited by: SS2.
* Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. Danning (2018)Hotpotqa: a dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369-2380. Cited by: SS2.
* Weihao Yu, Z. Jiang, Y. Dong, and J. Feng (2020)Reclor: A reading comprehension dataset requiring logical reasoning. In ICLR 2020 : Eighth International Conference on Learning Representations, External Links: Link Cited by: SS2.
* J. Zhang, L. Wang, R. K. Lee, Y. Bin, Y. Wang, J. Shao, and E. Lim (2020)Graph-to-tree learning for solving math word problems. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3928-3937. Cited by: SS2.
* B. Zhou, D. Khashabi, Q. Ning, and D. Roth (2019)"going on a vacation" takes longer than "going for a walk": a study of temporal commonsense understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 3363-3369. External Links: Link Cited by: SS2.

## Appendix A Discourse Delimiter Library

Our discourse delimiter library consists of two parts, the "Explicit" connectives annotated in Penn Discourse TreeBank 2.0 (DPTB 2.0) Prasad et al. (2008) as well as a set of punctuation marks. The overall discourse delimiters used in our method are presented in Table 4.

## Appendix B Implementation Details

We fine-tune RoBERTa-Large Liu et al. (2019) as the backbone pre-trained language model for DGAN, which contains 24 hidden layers with hidden size 1024. The overall model is end-to-end trained and updated by Adam Kingma and Ba (2015) optimizer with an overall learning rate of 5e-6 and a weight decay of 0.01. The overall dropout rate is 0.1. The maximum sequence length is 256. We tune the model on the dev set to obtain the best iteration steps of graph reasoning, which is 2 for ReClor data, and 3 for LogiQA data. The model is trained for 10 epochs with a batch size of 16 on Nvidia Tesla V100 GPU.

For the answer prediction module, the hidden size of GRU is the same as the token embeddings in the pre-trained language model, which is 1024. The two-layer perceptron first projects the concatenated vectors with a hidden size of 1024 \(\times\) 3 to 1024, then project 1024 to 1.

\begin{table}
\begin{tabular}{|c|} \hline
**Explicit Connectives** \\ 'once’, ‘although’, ’though’, ’but’, ’because’, \\ 'nevertheless’, ‘before’, ’for example’, ’until’, ‘if’, \\ ’previously’, ’when’, ‘and’, so’, ’then’, ’while’, ‘as long as’, ’however’, ‘also’, ‘after’, ‘separately’, ‘still’, ’so that’, \\ ‘or’, ’morever’, in addition’, ’instead’, ’on the other hand’, ‘as’, for instance’, ’nonetheless’, “unless’, \\ ‘meanwhile’, ’yet’, ‘since’, ’rather’, ‘in fact’, ’indeed’, \\ ’later’, ‘ultimately’, ’as a result’, ’either or’, ‘therefore’, \\ ‘in turn’, ‘thus’, ’in particular’, ‘further’, ‘afterward’, \\ ‘next’, ‘similarly’, ‘besides’, if and when’, ’nor’, \\ ‘alternatively’, whereas’, ‘overall’, by comparison’, \\ ‘till’, ’in contrast’, ’finally’, ‘otherwise’, — as if’, ’thereby’, \\ ’now that’, ‘before and after’, ‘Additionally’, ’meantine’, \\ ‘by contrast’, ‘if’ then’, ‘likewise’, ‘in the end’, \\ ‘regardless’, ‘the thereafter’,‘earlier’, ‘in other words’, ‘as soon as’, ‘except’, in short’, ‘neither nor’, ‘fulthermore’, \\ ‘lest’, ‘as though’, ‘specifically’, ‘conversely’, \\ ‘consequently’, ‘as well’, ’much as’, ‘plus’, ‘and’, \\ ‘hence’, ’by then’, ’accordingily’, on the contrary’, \\ ‘simultaneous’, ’for’, ‘in sum’, ‘when and it’, ‘isomfar as’, ‘else’, ‘as an alternative’, ‘on the one hand on the other hand’ \\ \hline \hline  & **Punctuation Marks** \\ \hline \multicolumn{2}{|c|}{} \\ \cline{2-2} \end{tabular}
\end{table}
Table 4: The discourse delimiter library in our implementation.