discopy: A Neural System for Shallow Discourse Parsing (2021)

Abstract: This paper demonstrates discopy, a novel framework that makes it easy to design components for end-to-end shallow discourse parsing. For the purpose of demonstration, we implement recent neural approaches and integrate contextualized word embeddings to predict explicit and non-explicit discourse relations. Our proposed neural feature-free system performs competitively to systems presented at the latest Shared Task on Shallow Discourse Parsing. Finally, a web front end is shown that simplifies the inspection of annotated documents. The source code, documentation, and pretrained models are publicly accessible.

## introduction whenever people compose text, they (consciously or not) make sure that related sentences are cohesive and coherent within a common section. by using models to analyze discourse, we identify relations within a text that consist of phrases and a certain sense. these discourse relations and their understanding are important for other tasks in the nlp community such as abstractive summarization @xcite ), text simplification @xcite , and argumentation mining @xcite . the penn discourse treebank (pdtb) @xcite , for example, describes discourse as a set of individual relations. the following two (artificial) examples demonstrate the main types of relations annotated in their corpus: this work introduces an end-to-end neural system that implements recently researched components for these tasks. our goal is to design components that all rely on the same contextualized word embeddings as input and, thus, it avoids the necessity to train huge neural networks. we compare our results with state-of-the-art shallow discourse parsers that took part at the conll shared task in 2016 @xcite which is to our knowledge the last time full systems were published. the contributions of this paper are as follows: ## system architecture in our framework discopy, we design a clean and modular parser pipeline principle. each component has to implement a specified interface that assumes to process a well defined document structure. starting with a document without annotated discourse relations, each component further adds its predictions while the document is passed through the pipeline, for example one component identifies connectives and their sense and, based on this prediction, another component extracts arguments around these connectives to complete the explicit relations. this parser architecture makes the definition of a custom pipeline easy and components remain interchangeable. together with the framework, we provide multiple components for an endto-end neural shallow discourse parser. the simple pipeline interface (exemplified in listing 2) with methods such as fit, parse, and eval, ensures pleasant user experience. the system combines recent research on various subtasks with advances in contextualized word embeddings. we make the framework and pre-trained components available under https://github.com/rknaebel/discopy . in the following, we briefly summarize the components implemented in our system. ## visualization in conjunction with our system, we provide a prototypical web-based front end with several views that simplifies the inspection of parser results. it can be used to visualize annotated documents, parse documents, and inspect them interactively. further, the visual inspection makes it easier to identify weaknesses of a developed parser, e.g. inaccurate connective identification or misplaced argument spans. the system is built on top of a simple python web service powered by fastapi foot_1 and it is designed using a reactive javascript library called vuejs foot_2 . both sides communicate via restful interface with each other. figure 1 demonstrates the view of a selected document. at the top, there is a simple search bar for easily accessing documents. directly below, the document's text is shown together with a list of its discourse relations at the bottom of the view. then, each discourse relation is listed with its corresponding relation type (e.g. explicit, implicit, entrel, altlex), as well as its sense. different parts of a relation, first and second arguments as well as the connective, are highlighted by colors. additionally, some context tokens are given before and after the relation if possible. ## experiments the connective sense classifier takes the connective candidate and one surrounding context tokens as input. the two hidden layers both have dimensions 256. a dropout layer follows after each hidden layer with a 0.3 drop rate. both argument extractors use two bi-directional recurrent layers with a hidden size of 256 in each direction, a dropout rate of 0.2 before and after the fully connected layer having a dimension of 128. the explicit extractor is trained on explicit examples with a window size of 100. the implicit sense classifier accepts two arguments with at most 35 tokens each. the same bi-directional recurrent layer with hidden size 128 processes both arguments. the drop rate of the dropout layer that follows after concatenation is 0.25, and the size of the hidden dimension afterward is 128. all models are trained with amsgrad adam @xcite and a 0.001 learning rate. ## results and discussion for our evaluation, we use the same experimental design as proposed at the conll shared task 2016 @xcite . we have trained multiple pipelines with varying language models used to compute the input embeddings. the model names in our experiment tables follow the naming of the word embedding model foot_3 . as comparison systems, we choose the overall best performing submissions ## conclusions in this work, we demonstrated a framework for shallow discourse parsing and integrated a web-based graphical user interface to study parser outputs. we presented an exemplary pipeline built from neural components that are based on recent models. we extend previous approaches to incorporate precomputed contextualized word embeddings. our pipeline system performs competitively to former discourse parsers and partially outperforms them, while not using any linguistic features. in the future, we plan to integrate more recent neural architectures into the system to improve overall scores. further, we want to improve the graphical interface to seamlessly integrate the visualization directly into the text such as done with brat foot_4 for dependency parses. thus, it will be easier to compare multiple relations in their context. overlapping arguments are a major problem, why we decided to list relations separately. building on that a view would be useful for comparing discourse relations across documents, e.g. comparing predictions and gold annotations or multiple parser outputs.