D is C ut and D isc R e T : MELODI at DISRPT 2023 (2023)

Abstract: This paper presents the results obtained by the MELODI team for the three tasks proposed within the DISRPT 2023 shared task on discourse: segmentation, connective identification, and relation classification. The competition involves corpora in various languages in several underlying frameworks, and proposes two tracks depending on the presence or not of annotations of sentence boundaries and syntactic information. For these three tasks, we rely on a transformer-based architecture, and investigate several optimizations of the models, including hyper-parameter search and layer freezing. For discourse relations, we also explore the use of adapters—a lightweight solution for model fine-tuning—and introduce relation mappings to partially deal with the label set explosion we are facing within the setting of the shared task in a multi-corpus perspective. In the end, we propose one single architecture for segmentation and connectives, based on XLM-RoBERTa large, freezed at lower layers, with new state-of-the-art results for segmentation, and we propose 3 different models for relations, since the task makes it harder to generalize across all corpora.

## introduction discourse analysis consists in building a discourse structure representing the organization of a document -a monologue or dialogue -, as the discourse tree in figure 1 . first, the document is split into minimal sub-units, called elementary discourse units @xcite : the text in the example, consisting of two sentences, is divided into 5 edus (from 2 to 6). the edus are then attached together, forming larger discourse units -such as the pair (edu2, edu3) -that are recursively linked to form a tree or a graph, depending on the underlying framework. the links between the discourse units are semantic-pragmatic relations, such as conces-sion, evidence, sequence etc. these relations can be triggered by an explicit lexical item, a connective such as because, while, or when for condition in the example. relations can also be "implicit", when no such marker is present, such as the concession between edu2 and edu3. there are mainly three frameworks for discourse: rhetorical structure theory (rst) @xcite ) -from which the example in figure 1 is derived -, segmented discourse theory (sdrt) @xcite -where structures are graphs -, and the penn discourse treebank (pdtb) @xcite , where discourse relations are sparsely annotated without constraints on the overall structure. alternatively, there have been proposals to transform discourse structure into simpler dependency structures (dep), e.g. in rst @xcite or sdrt @xcite . recently, this view has been taken to annotate directly new data in the scidtb corpus @xcite , proposing a set of relations and segmentation rules inspired by rst but producing trees of dependency relations between edus. several corpora have been annotated under each framework for different languages: however, even within the same framework, annotation guidelines and relation sets might be different for each corpus. the disrpt shared task intends to provide a unified format for researchers to evaluate their systems against varied languages, domains, and frameworks. three tasks were proposed: (1) discourse segmentation into edus, (2) identification of discourse connectives, and (3) classification of discourse relations based on attached units. the first two tasks are encoded with a bio scheme over tokens, the latter corresponds to a multi-class classification between pairs of textual segments. the benchmark provided within disrpt allows us to verify the robustness of our approach through 13 languages, 4 frameworks, and varied domains, including multi-party dialogues and speech transcriptions. in this paper, we address the three tasks through two systems: discut foot_0 for tasks (1) and ( 2 ) and discret foot_1 for task (3). these systems both rely on transformer architectures and we thoroughly investigate different variations of the pre-trained model and the hyper-parameters values, while also varying the level of frozen layers. this latter parameter allows for lighter models, and also improvements in most cases. for task (3), we also investigate adapters @xcite that provide a lightweight solution for transferring to new tasks. for all tasks, we favor multilingual pretrained models, in order to better generalize and experiment with corpus merging for relations, with the aim of providing a generic model that can be used for any corpus. in the end, we ranked first on discourse segmentation on the treebanked track (+0.87 on the average, compared to the other system) but second for connectives (-0.47), and we are the only system with results on the plain track, with higher performance than the winner of disrpt 2021. for relations, our system is the only one trying to mix all corpora, thus even if the performance are lower than other proposed approaches, it is possibly better at generalization. ## related work discourse parsing is the task of building the full trees/graphs. most work focuses on attachment or discourse relation identification, and on english. recently, a multilingual rst discourse parser has been proposed @xcite , building on previous work @xcite but proposing to jointly learn attachment and edu segmentation and adding a cross-lingual strategy, rather than english only. it shows that multilingualism is a key component to improve performance, since data scarcity affects even english, and that good segmentation is crucial, with a loss of up to 8% with predicted edus for full parsing. discourse segmentation was considered a solved task, with scores as high as 94% @xcite , but it was later shown that performance drops for languages other than english, -linked to smaller corpora and lesser resources -, and when gold sentences are not given, due to sentence segmenters far from being perfect @xcite . the first edition of the disrpt shared task @xcite also revealed the same trend with performance above 95% for some corpora, but also issues with others such as the spanish scdt (82.5% at best) or the russian rrt (86.2%). the bestperforming system in 2019 @xcite was using a single model based on multilingual bert for every corpus @xcite , while in the second edition @xcite , the best system @xcite relied on varied language models, either mono-or multilingual, associated to hand-crafted features: best overall performance was around 91.5% on average, with a loss of about 2% when the sentences are not given. connective identification was first seen as a word disambiguation task, where the goal was, starting with a list of candidates, to decide whether each occurrence is used in a discourse reading or not @xcite . it has been then recast as a sequence labeling one, where we need to decide whether a token starts, is within, or is outside a discourse connective @xcite . as for segmentation, performance drops when existing systems are trained on new domains or languages @xcite , but fewer studies investigated this task since implicit relations are more an issue for discourse parsing. the first two editions of disrpt demonstrated rather high performance: between 92 -94 for the english and turkish corpora, and 87 for the chinese one, with only a small drop when sentences are not given. discourse relations are the main object of study within the domain, with a specific focus on implicit ones since the connective is considered a very strong clue for guessing the relation @xcite . however, again, performance drops, even for explicit relations when data are scarce @xcite . moreover, a real-life scenario has to deal with both implicit and explicit relations, it is thus interesting to see results combining all types of relations, and for several languages. only two systems were presented in 2021, and the winning model was based on transformers, with a specific pretrained model depending on the target language and additional hand-crafted features: best overall performance is still low, with 61.8%. ## data the 2023 disrpt shared task, including suprise datasets, provides 26 corpora for 13 languages and 4 theoretical frameworks: 9 correspond to the pdtb framework (thus connective and relations), the others are either rst (12), dependency (3) or sdrt (2) (thus segmentation and relations). among these, 10 new corpora are introduced in the 2023 edition: 6 are released as surprise datasets, with one new language (thaï), and out-of-domain (ood) data for english (covid-dtb and ted), portuguese (crpc and ted) and turkish (ted). all statistics are given in we have 6 corpora for english @xcite @xcite @xcite , 4 for chinese @xcite @xcite , 2 for spanish @xcite , 2 for portuguese @xcite , 1 for german @xcite , 1 for basque @xcite , 1 for farsi @xcite , 1 for french @xcite , 1 for dutch @xcite , 1 for russian @xcite , 1 for turkish @xcite , 1 for italian @xcite and 1 for thai. in addition, ood datasets come from the multilingual ted discourse bank with data for english, portuguese and turkish @xcite . the model is based on a pretrained language model (lm), with an additional linear layer for token classification. the lm is multilingual, allowing it to be used for all corpora. contrary to systems proposed in 2019 and 2021 based on a similar architecture, we removed the cnn at the character level, and the lstm outer layer, as additional experiments demonstrated no improvements. the lm is based on a transformer architecture with several layers within the encoder. it has been shown that, broadly speaking, lower layers mostly encode morpho-syntactic information, while upper contain more semantic ones @xcite @xcite . we thus experiment with freezing some lower layers while continuing the fine-tuning on higher levels, in order to have lighter models. "freezing a layer" is the process of disallowing the update of weights for the target layer during the finetuning process, meaning that the layer preserves its learned information from pretraining. models are fed with sentences, the documents being too long for the lms. we detail below our setting when sentences are not given ('plain' track). ## conclusion in this paper, we presented our submissions for the three tasks of the disrpt shared task. our main goals were to rely on only a few architectures variants for generality, and experiment with parameter efficient methods. for tasks 1-2, we employed multi-task, multi-corpora approaches; however, at this stage of our research our results are not opti- in parenthesis is the number of epochs for which the model was trained. 'discodisco' was the best-performing model of disrpt 2021 @xcite and 'diff.' is the comparison with our models. mean (all) provides the mean for the currently available datasets, while mean (2021) averages only disrpt 2021's corpora.