Neural-based RST Parsing And Analysis In Persuasive Discourse (2021)

Abstract: Most of the existing studies of language use in social media content have focused on the surface-level linguistic features (e.g., function words and punctuation marks) and the semantic level aspects (e.g., the topics, sentiment, and emotions) of the comments. The writer’s strategies of constructing and connecting text segments have not been widely explored even though this knowledge is expected to shed light on how people reason in online environments. Contributing to this analysis direction for social media studies, we build an openly accessible neural RST parsing system that analyzes discourse relations in an online comment. Our experiments demonstrate that this system achieves comparable performance among all the neural RST parsing systems. To demonstrate the use of this tool in social media analysis, we apply it to identify the discourse relations in persuasive and non-persuasive comments and examine the relationships among the binary discourse tree depth, discourse relations, and the perceived persuasiveness of online comments. Our work demonstrates the potential of analyzing discourse structures of online comments with our system and the implications of these structures for understanding online communications.

## introduction researchers are increasingly interested in analyzing the language used in social media content. previous studies examine surface-level linguistic features such as the use of function words or punctuation marks in online communications @xcite , and/or semantic level aspects such as understanding the sentiment @xcite and emotions @xcite as well as identifying topics @xcite . recently, studies also conduct discourse-level language analysis such as exploring cohesion in the social media content @xcite ) and using discourse level structure for detecting fake news @xcite . the pragmatic use of the language in the content is also limitedly explored, such as the analysis of imperative statements in wikipedia discussions @xcite . arguing for the importance of considering context information in language analysis, @xcite call for more nlp research that "develop tools to automatically determine the structure of discourse, including discourse relations, argumentation, and threads in conversations such as those found in twitter and other social media" @xcite . as a response to this call, we have developed a discourse parsing system that automatically constructs the discourse parsing tree for the input text. discourse parsing is a task of parsing a text segment into a set of discourse units and logically connect the merged units using a set of discourse relations, which exhibit reasoning behaviors in online environments @xcite . rhetorical structure theory (rst) @xcite is one of the most influential theories in discourse parsing and is used in our system. to parse rst, we need to build up a tree, where the nodes bear information such as span, nuclearity, and relation. the leaf node in the rst tree represents an elementary discourse unit (edu), a sequence of which could be merged into a longer span. the span place in the non-terminal nodes attaches with the nuclearity and relation labels. nuclearity type typically includes nn, ns and sn, is the semantic role in discourse, where the nucleus span (n) is more essential than the satellite span (s). the neural-based system of @xcite utilizes granularities including paragraphs, sentences and edus levels. the paragraphs and sentences granularities forcedly split a span at the paragraph/sentence boundary instead of other positions, resulting in the wrong subspans. there are around 7.6% edus with inconsistent sentence or paragraph index in one edu. in addition, they parse rst in a top-down manner, which results in a larger search space. for instance, there are n -1 possible split points for a span composed of n edus. they also add a second-layer blstm upon all the edus to view bidirectional context. building on their work, we conduct the following tasks to achieve a higher performance in rst parsing: to demonstrate the use of our discourse parsing system for analyzing discourse relations in social media content, we apply it in online persuasion research in two online contexts, namely, wikipedia afd and reddit cmv discussions. we investigate the differences between persuasive and non-persuasive comments regarding their discourse structure. our statistical analyses show that a comment's perceived persuasiveness is correlated to discourse relations and the depth of its binary discourse tree. not only showing the potential of our discourse parsing system for social media content analysis, but these findings also suggest that studying the discourse structure of social media content achieves a better understanding of complex online interactions such as persuasion. ## related works previous studies in rst parsing have two dominant parsing approaches including transition-based and neural-based approaches. the transition-based systems that rely on hand-crafted features have achieved great success @xcite ; while neural-based approaches do not require feature engineering, which does not take into account the particular nature of a specific data. networks such as attention-based hierarchical neural network @xcite , and pointer networks @xcite are developed to parse rst at sentence or text level. however, neither do neural network approaches consider word position and nor they design components for different nuclearity types, as initial and end words and nuclearity type are essential in rst parsing @xcite . the discourse parsing tools provide a new perspective in discourse interpretation in social media @xcite . @xcite conduct causal explanations in physical and mental health building on discourse parsing; @xcite develop a novel discourse annotation schema that reflects a hierarchy of discursive strategies for the analysis of contentious and polarizing discussions. discourse parsing also is leveraged in sentiment analysis @xcite , identify authorship @xcite and detect deception @xcite . in this paper, we build up a new comparable discourse parsing tool for the contribution to social media content analysis in the discourse aspect. ## rst parsing system in this section, we present our rst parsing system. we propose a neural rst parsing system in the bottom-up manner utilizing edu granularity. we emphasize the role of word positions, investigate the role of an additional layer of blstm, and design different relation submodels based on different nuclearity types. ## discourse relations in social media content: case studies we apply our rst parsing system to identify the discourse structure of a comment in two online discussions, namely, wikipedia afd and reddit cmv discussion datasets. these two datasets have been used in online persuasion studies @xcite . both datasets have comments labeled as persuasive or non-persuasive. we choose these datasets because we are interested in whether and how the discourse structure of a comment correlates with its perceived persuasiveness. persuasion is commonly considered as "human communication that is designed to influence others by modifying their beliefs, values, or attitudes" @xcite . in iyer et al. (2017) 's study, the researchers found that the syntactic structure of a sentence plays an important role in indicating different persuasion tactics. however, the discourse structure of a com-ment remains unstudied. the first step in our analysis is to segment the comment to edus for the study. we segment a comment by sentence, i.e., the edu in our case study is a sentence. comments that only have one sentence are not included in the analysis. we identify the discourse structure of a comment in these two datasets using our rst parsing system and then investigate the relationships among a comment's binary discourse tree depth, discourse relations, and perceived persuasiveness. we refer to binary tree depth as shallow and deep with the value of 0 and 1 respectively in the following sections. in these discourse structure analyses, we divide the 18 raw discourse relations into three groups according to the definition of presentational, sub-ject_matter, and multinuclear relations proposed by @xcite . of these three types of rst relations, we consider presentational relations to have the strongest persuasion intent as the intended effect of these relations to increase some inclination in the reader such as the reader's degree of agreement or positive view regarding the presentational background, enablement, summary, explanation subject_matter attribution, cause, condition, elaboration, evaluation, manner-means,topic-comment multinuclear comparison, contrast, joint, same-unit, temporal, textual-organization, topic-change ## discussion complex social interactions increasingly happen through social media @xcite ). text's discourse structure reflects how the writer constructs arguments and reasons, which is expected to help us better understand the dynamics of these social interactions. through the analysis of the comment's discourse structure in the two online discussions and the finding that a comment's discourse structure, identified from the tool, correlates with the comment's perceived persuasiveness, we showcase the potential of using this tool in social media content analysis. essentially, the availability of this tool offers new opportunities for social media research. how does a comment's discourse structure correlate with the amount of attention it receives, in twitter, facebook, or reddit discussions? do people converge at the level of discourse structure as the online discussion goes on? do toxic comments or disinformation tend to have particular discourse structures? these are a few example research questions for these new opportunities. our tool currently takes one sentence as an edu. one of our future tasks is to parse a comment into smaller discourse units and build an rst tree upon them. also, our model is trained on wall street journal (rst-dt) @xcite , the context and the text length of which is less similar to social media communications. the token number in a sentence in wall street journal (around 18 tokens) is much less than that in afd (around 63 tokens) and in cmv (around 246 tokens). we are exploring other rst annotated datasets as the training data for our future model. our case study of the comment's discourse structure in wikipedia afd and reddit cmv discussions contributes to online persuasion research. the findings of our statistical analyses demonstrate that at the discourse structure level valuable information can be obtained that reflects the persuasion dynamics and helps interpret the perceived persuasiveness of online comments. additionally, the contextual influence on social media communications can be explored at this level of language analysis as well. we also try to apply the machine learning model proposed by @xcite to the case study. the model proposed by @xcite achieves the micro f1 score of 87.9, 74.9, 62.6 in span, nuclearity, and relation. the findings remain the same after applying both their and our models, indicating that the performance of an rst parser has little influence on the correlations we reveal in these two online environments. we call for more investigations on how discourse relations and perceived persuasiveness interact in different online communities contexts. ## conclusion various natural language processing techniques have been applied in social media content analysis such as word frequency measures, syntactic structure analysis, sentiment analysis, emotion recognition, or topic modeling. recently, researchers call for further consideration of contextual factors in the analysis and suggest the exploration of the discourse structure @xcite . in response to this call, we build a bottom-up transition-based system for discourse parsing using the rhetorical structure theory (rst) framework. in this system, we consider the position of tokens in an edu and we pay partial attention to an edu regarding the whole semantic of an edu. we also design relation submodel based on nuclearity types. our final model achieves the highest performance among all the reported neural networkbased systems for rst parsing. we find that the low-frequency relations (please check appendix) such as topic-change and topic-comment do not achieve satisfying performance. in addition, these relations might benefit from global information. therefore, we plan to deal with the imbalanced data by paying more attention to the low-frequency relations, and try to embed global information in the bottom-up parsing system in the future. to demonstrate the use of our model to conduct discourse structure analysis in social media content, we explore the relationships among a comment's binary discourse tree depth, the discourse relations it contains, and its perceived persuasiveness in two online discussion contexts: wikipedia afd and reddit cmv discussions. our analyses show that the depth of a comment's binary discourse tree and its discourse relations are correlated and independent of the discussion context. in wikipedia afd discussion, comments that have subject_matter relations are more likely to be perceived as persuasive, while in reddit cmv discussions, comments constructed by presentational and multinuclear relations are perceived to be more persuasive.