DSNDM : Deep S iamese Neural Discourse Model with Attention for Text Pairs Categorization and Ranking (2020)

Abstract: In this paper, the utility and advantages of the discourse analysis for text pairs categorization and ranking are investigated. We consider two tasks in which discourse structure seems useful and important: automatic verification of political statements, and ranking in question answering systems. We propose a neural network based approach to learn the match between pairs of discourse tree structures. To this end, the neural TreeLSTM model is modified to effectively encode discourse trees and DSNDM model based on it is suggested to analyze pairs of texts. In addition, the integration of the attention mechanism in the model is proposed. Moreover, different ranking approaches are investigated for the second task. In the paper, the comparison with state-of-the-art methods is given. Experiments illustrate that combination of neural networks and discourse structure in DSNDM is effective since it reaches top results in the assigned tasks. The evaluation also demonstrates that discourse analysis improves quality for the processing of longer texts.

## introduction the growing popularity of social networks and the widespread use of social media contributed to the emergence of many nlp tasks associated with the processing of statements. it can be analyzed from an emotional point of view (sentiment analysis), opinion and argumentation mining, text summarization and so forth. despite the success of the transformer-based neural networks, such as bert @xcite and its modifications, in various nlp tasks, they also have disadvantages since they frequently analyze only the plain text that can be quite long and complex. at the same time, discourse structure contains important knowledge for solving these tasks, and several researchers demonstrated its significance @xcite @xcite . however, the value of discourse have been already investigated only for some single text categorization tasks. in this study, we demonstrate the utility and advantages of the matching of discourse tree structures of text pairs. discourse analysis seems effective in textual entailment, text simplification and paraphrase detection tasks. however, it is necessary to analyze texts on the sentence level in most cases. we consider typical nlp tasks in which input texts are quite long and paragraphs are given initially. one of such tasks is automatic verification of factual texts. politicians may utilize unreliable statements for their own purposes. due to the fact that there are plenty of such statements, it should be automatically evaluated for reliability and the possibility of manipulation of public opinion. in most cases, it is possible to extract some confirmation or refutation for a given factual text. in this way, we investigate the utility of discourse analysis in the classification of pairs of texts: statements and their justifications. discourse structure may contain crucial knowledge even for the classification of the statements alone but can be even more effective in the case of analyzing additionally the confirmations and refutations. apart from that, one of the most appropriate tasks is the ranking in question answering systems. it was shown that discourse structure of questions and correct answers should correlate @xcite . companies are interested in qa systems development in order to maximize the ease of interaction with customers. all questions can be divided into two groups: factoid and non-factoid. it is important to answer factoid questions to provide some specific information and non-factoid ones to maintain a dialogue. it is worth to emphasize that the second task is more challenging because there is no single correct answer for each question. we consider the non-factoid questions asked on internet forums since the discourse analysis seems to be more helpful in this case. the main technical idea of this paper is to combine discourse analysis and recursive neural network treelstm @xcite , which previously obtained the state-of-the-art results in some single text classification tasks. our contributions can be formulated as the following: our paper is organized as follows. firstly, we summarize related work and introduce some base concepts. we continue with the description of the base model and its modifications. then, we discuss the obtained results, error analysis and propose directions for further research. ## related work there are several approaches to solve the factchecking problem. the best models presented in the fever competition @xcite allocate a stage of extracting supporting or refuting information and a classification stage. justifications have already been extracted in our case. therefore, there is no need to use the first stage. the bert model @xcite is frequently used as the main model of the approach @xcite . it is worth to emphasize that bert cannot process long texts (the sequence is limited to 512 tokens). therefore, it is necessary to extract the key information from the given justification paragraph. besides, bert can not efficiently store and process discourse features. another approach uses knowledge graphs. @xcite proposed the use of relations between the entities of the graph in order to confirm some "distill" information extracted from the statement. @xcite suggested knowledge linker, the main idea of which is that if the path between entities in the knowledge graph is short, then the factual text containing them is reliable. it should be mentioned that this approach is generally applicable only to factoid statements since the entities must exist within knowledge graphs. finally, we distinguish the third approach which considers structural information extracted from texts @xcite . @xcite proposed to match discourse trees and solved the categorization task using tree kernel-based svm. however, this approach does not utilize any modern neural networks. at the same time, recursive neural networks are gaining popularity @xcite @xcite . the main goal of them is to encode tree-like structures, such as syntax and discourse trees. these models achieved superior results in the single text categorization tasks, but researchers did not investigate the value of discourse analysis for processing pairs of texts. however, this approach is promising for the assigned task since frequently not only unreliable texts have a similar discourse structure, but the discourse structure of texts refuting them is also similar. the main baselines for the question-answering problem are models that utilize keywords for ranking: using tf-idf, bm25 and its modifications (okapi bm25, bm25f). frequently, their results are bad enough and need to be re-ranked using more complex methods. neural network models, such as bert, allow obtaining state-of-the-art results @xcite . it should be mentioned here that different training techniques of ranking are often not investigated. in addition, some fact-checking approaches can be applied in question answering systems. for instance, @xcite and @xcite considered the possibility of using knowledge graphs. @xcite investigated the value of discourse analysis in qa systems, but did not utilize any neural network approaches. ## conclusion and future work in this paper, we investigated the utility and importance of the discourse analysis for text pairs categorization and ranking. we considered two typical tasks in which discourse analysis seems promising: automatic verification of political statements and ranking in question answering systems. we modified treelstm to effectively encode discourse trees and proposed dsndm which is capable of processing pairs of texts. in addition, the integration of the attention mechanism in the proposed model was suggested to obtain more useful embeddings of subtrees. moreover, we investigated three training techniques for the ranking task. the experiments were performed on the liar-plus and antique datasets. dsndm efficiently learned the match between discourse tree structures and achieved high quality in both tasks. besides, the attention module improved the metrics of the base model in all cases. the error analysis showed that the model processes deeper trees more successfully. there are possible directions for future work: the use of trees not only of a binary structure, the modification of vector representations of edus, as well as the investigation of the performance of dsndm in other various tasks where discourse analysis may be helpful, e.g. machine translation, chat-bots and other qa systems. apart from that, we will experiment with other hierarchical structures (e.g. syntactic) for deeper analysis of the importance of the rst-based structure in the proposed model. hallock said warner, in this fall's senate election, ran 8 -10 points ahead of past performances by fellow democrats mcauliffe and kaine in rural virginia. mcauliffe's portion of the rural vote, in his 2013 gubernatorial victory, was 3.6 percentage points below warner's. kaine's slice of the rural vote, in his 2012 senate win, was 2.4 percentage points above warner's. in the u. s. constitution, theres a little section in there that talks about life, liberty and the pursuit of happiness. no court makes a legal decision based on the declaration of independence, wilkes said. with his first speech as a bona fide candidate, cain joins a long, bipartisan line of presidential hopefuls who have succumbed to foot -in -mouth disease. they include cain's foe, president barack obama, who accidentally said there were 57 states during the 2008 campaign and u. s. sen. john mccain, who said in an interview he was unsure how many houses he owned. welcome to the 2012 presidential election season, folks. false in the illinois legislature, barack obama voted present, instead of yes or no on seven votes involving abortion rights. two other large groups, naral prochoice america and planned parenthood, are not endorsing. planned parenthood, however, has given both candidates 100 percent ratings for their records on abortion. we stipulate that there are clearly different interpretations of the significance of obama s present votes. but there s no doubt he made them. true as a result of climate change, ice fishermen in wisconsin are already noticing fewer days they can be out on our ice covered lakes. not what is going to happen this year. our rating it's been a longer and colder winter than in recent years. but that doesn't erase a trend that's been well -established. the number of days that the lakes have ice on them -making them safe for ice fishing -has declined. true