EDTC : A Corpus for Discourse-Level Topic Chain Parsing (2021)

Abstract: Discourse analysis has long been known to be fundamental in natural language processing. In this research, we present our insight on discourse-level topic chain (DTC) parsing which aims at discovering new topics and investigating how these topics evolve over time within an article. To address the lack of data, we contribute a new discourse corpus with DTC-style dependency graphs annotated upon news articles. In particular, we ensure the high reliability of the corpus by utilizing a two-step annotation strategy to build the data and filtering out the annotations with low confidence scores. Based on the annotated corpus, we introduce a simple yet robust system for automatic discourse-level topic chain parsing.

## introduction topic information as a crucial auxiliary for text understanding has drawn great attention in recent decades @xcite @xcite . in the literature, previous studies on topic modeling usually extract topics by introducing latent variables for tokens for topic assigning @xcite @xcite . similarly, researches on text-tilling achieve topic segments through lexical cohesion modeling @xcite . instead of lexical cohesion measuring, @xcite put their attention on evaluating the organization and cohesion of pieces of evidence and build topic chains on related text units. besides, recent studies on argument mining explore to build links or clusters for topic-dependent arguments @xcite @xcite . obviously, more and more researches show that there are certain structures among topic segments that deserve deeper exploration. in this work, we aim to explore the cohesion of topic-related text segments. different from @xcite , we show great interest in uncovering how fine-grain topics emerge, evolve, and disappear in an article, which is referred to as discourselevel topic chain (dtc) parsing. since the dtc structure can provide relatively rich and low-noise information about certain topic aspects of articles, it is meaningful for various nlp tasks like summarization @xcite , document similarity measuring @xcite , and response generation @xcite . in the literature, topic detection and tracking (tdt) @xcite is a research area most similar to dtc parsing which aims at identifying new events and tracking how they change over time. however, the events in the tdt task refer to happenings at certain places and times which only compose a small subset of general topics. recently, @xcite manually annotate the first chinese dtc corpus based on the theme-rheme theory @xcite . by contrast, due to the lack of corpus, previous study on english dtc parsing usually uses unsupervised methods @xcite to explore the structure and trends of important topics hidden within news articles. obviously, one intractable problem facing dtc parsing is the lack of data. this research is primarily motivated by @xcite on the topic chain concept, @xcite on dtc corpus construction, and @xcite on topic-dependent argument linking. and our contributions mainly include two aspects: (i) building an english corpus of discourse-level topic chain (edtc) through a two-step annotation method and (ii) lunching a simple but robust bert-based baseline system for automatic dtc parsing. moreover, as implied in recent researches on discourse rhetorical structure (drs) parsing @xcite @xcite , discourse parsing remains challenging due to the lack of data. under this circumstance, we annotate the dtc structures for the 385 wall street journal (wsj) articles in the rst-dt @xcite corpus aiming to build a bridge between discourse rhetorical structure and dtc structure for discourse researchers to utilize. ## corpus annotation before detailing the annotation process, we give a formal introduction to the "topic" mentioned in this paper. in topic modeling, a topic is usually viewed as a probability distribution over a fixed vocabulary @xcite . in addition, previous studies on argument mining usually manually define some coarse-grain topic categories for either topic-dependent argument classification or clustering @xcite . different from previous work, topics in this study refer to fine-grained topic categories that fit the context. for example, given the sentence "house prices are expected to be fragile.", the coarse-grained topic label of it could be "economics" and the fine-grained label is "house price". comparing the two kinds of labels, the first one seems more like the theme of an article which is useful in text-clustering or text-tilling, and the second one gives us more detailed description on the topic itself which is more practical in discourselevel topic chaining. for better understanding of our annotation, we present some preliminary definitions as following: discourse topic unit (dtu) refers to the elementary topic unit in our annotated dtc structure. in the literature, @xcite hold the view that each sentence is composed of multiple dtus with different sub-topics which they refer to as elementary discourse topic unit (edtu). different from them, we study macro dtc structures in this work where each sentence is taken as an independent dtu 1 . it is worth mentioning that not all the 1 although we built the corpus based on rst-dt, it remains dtus are topic-bearing, there are also some sentences with no topic meaning, e.g., the sentence "oops!". topic object (to) could be subject, object, or other noun or noun phrase in the dtu which can provide a certain basis for topic chain parsing. usually, each to is closely related to the topic of its dtu, and each dtu maintains an independent to set. notably, the "to" mentioned here is not directly equivalent to the "entity" in co-reference resolution, the judgment of to requires a comprehensive consideration of document context. for example, given the dtu "drexel burnham lambert inc. is the adviser on the transaction.", if the surrounding context of the dtu is mainly about the company, then we choose "drexel burnham lambert inc." as a to; if the context is mainly about the transaction, then we choose "transaction" as a to, and we can also select both of them if necessary. it is worth mentioning that the tos could also be implicit ones which require human judgments. topic event (te) refers to the main phrase which most clearly expresses an event occurrence or a description of the tos in the dtu. for the dtu u4 in figure 1 , we select "develop vaccines against the virus" as the topic event of the dtu. with the above-mentioned definitions in mind, we argue that each dtu is composed of a set of tos and a core te. based on this concept, we give the following four annotation suggestions: there is a certain textual structure in between the dtus from u2 to u8 (e.g., they share the multinuclear relation list in the rst theory @xcite ), but the topic transitivity is weak. therefore, we do not mark any topic chains among the dtus. here we take the example in figure 1 to illustrate the annotation process. simply put, the annotation process is also the process of comparing the to and tes of the current dtu with that of the previous ones. according to the annotation instructions, we do the comparison from near to far aiming to obtain the closest path for two adjacent dtus on the chain. for the dtu u1, its to set contains two topic objects, i.e., "coronavirus" and "covid-19", and its core topic event can be sketched as "coronavirus outbreak in wuhan". correspondingly, the to set of u2 contains a pronoun object "it", referring to "coronavirus", and its core te is manually detected as "there is still no idea how to beat it". obviously, the two to sets have an intersection (i.e., "coronavirus") and the te in u2 does evolve from that in u1. consequently, we mark a topic link between the two dtus. for u3, both the to set and te do not meet our annotation requirements, so we neither link it to u1 nor u2. for u4, the to set is relevant to that of u3 as international institutions and the two tes are also interrelated, we therefore build a link between them. in this way, the overall vein of topic chains will be built after several rounds of comparison. notably, from the resulting graph we find that the topic chain with u1, u2, u5, and u-k on it does provide rich and low-noise information about the evolution of covid-19, which reflects the practical value of our annotated dtcs. ## baseline recent years have witnessed the great effects of pre-trained language models @xcite ) method in our baseline system. given a discourse with k-1 dtus, we use the pretrained bert foot_0 model to encode the entire discourse where each dtu is surrounded by the [cls] and [sep] tokens. and we take the bert output corresponding to [cls] as our dtu representation. following previous work, we also fine-tuned the pre-trained language model parameters during the training process. for the convenience of calculation, a zero-initialized vector u z is added at the end of the dtu sequence for the tail dtus of the topic chains or the isolated dtus to point to, obtaining @xmath0, . . . , u k-1 , u z ). for dependency parsing, we simply build a bi-linear function between u and its duplicate to achieve it, as following: ## conclusion in this research, we explored how fine-grain topics emerge, evolve, and disappear within an article. to address the lack of data, we built an english dtc corpus through a two-step annotation method, and filtered out the annotations with low confidence scores to ensure the high reliability of the corpus. during annotation, we found that each annotated topic chain does provide relatively low-noise information about a certain aspect of the article and the complete dtc structure can well describe the overall vein of topics in an article. with this in mind, we introduced a simple and robust baseline system, and the parsing model we trained can be straightforwardly harnessed in downstream topic-sensitive nlp tasks to boost performance. it is worth mentioning that we annotated the wsj articles in the rst-dt corpus also aim to allow the discourse researchers to explore the potential correlation between rst-and dtc-style discourse analysis in future work.