Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning (2025)

Abstract: This paper introduces the first multi-lingual and multi-label classification model for implicit discourse relation recognition (IDRR). Our model, HArch, is evaluated on the recently released DiscoGeM 2.0 corpus and leverages hierarchical dependencies between discourse senses to predict probability distributions across all three sense levels in the PDTB 3.0 framework. We compare several pre-trained encoder backbones and find that RoBERTa-HArch achieves the best performance in English, while XLM-RoBERTa-HArch performs best in the multi-lingual setting. In addition, we compare our fine-tuned models against GPT-4o and Llama-4-Maverick using few-shot prompting across all language configurations. Our results show that our fine-tuned models consistently outperform these LLMs, highlighting the advantages of task-specific fine-tuning over prompting in IDRR. Finally, we report SOTA results on the DiscoGeM 1.0 corpus, further validating the effectiveness of our hierarchical approach.

## introduction discourse analysis explores how textual segments are connected through relations of coherence. one of the most widely adopted frameworks for computational discourse analysis is the penn discourse treebank (pdtb) @xcite . in the pdtb framework, a discourse relation is established between two textual segments (referred to as arguments) either explicitly, through the presence of a discourse connective (e.g., but and because), or implicitly when no connective is present. the sense of the relation is then classified using a defined list of discourse senses, which are organized hierarchically across three levels @xcite . the task of identifying the sense of an implicit relation is known as implicit discourse relation recognition @xcite . research on idrr has mostly been conducted on the english annotated pdtb corpora @xcite . however, the subjective nature of discourse interpretation poses challenges for single-label annotation schemes, where each instance is assigned only one sense label @xcite @xcite . the ambiguity in the annotation of implicit relations has been further highlighted by the challenges in mapping them across discourse frameworks @xcite . as a response, recent works have advocated for multi-label annotation, recognizing that multiple discourse senses can often co-occur within a single relation @xcite @xcite @xcite . this shift in perspective has led to the development of the multi-label annotated discogem 1.0 corpus @xcite for implicit discourse relations. this perspective shift toward multi-label annotation has stirred an initial wave of research in multilabel idrr @xcite @xcite . however, as with the pdtb corpora, the discogem 1.0 corpus is limited to the english language and most work in the field of discourse continues to focus mostly on the english language. while single-label discourse annotated corpora following the pdtb framework exist in other languages, such as the german potsdam commentary corpus 2.2 (bourgonje and @xcite and the czech prague discourse treebank 3.0 @xcite , multilingual corpora remain scarce. for instance, the ted-mdb corpus @xcite provides parallel annotations of single-label discourse relations across seven languages. however, its size remains relatively small for the training of idrr models -it contains 1, 774 implicit relations. to address this gap and enable cross-linguistic research in idrr, the recently released discogem 2.0 corpus @xcite introduces parallel multilabel annotations of implicit discourse relations ## previous work the advent of transformer-based models @xcite has led to substantial progress in natural language understanding. nevertheless, idrr remains one of the most challenging tasks in computational discourse analysis. traditionally formulated as a single-label classification problem, most approaches have addressed idrr in english either by fine-tuning @xcite or prompt-tuning @xcite @xcite pre-trained language models (plms). another line of research has also explored directly prompting large language models (llms) through prompt-engineering to solve single-label idrr @xcite . however, results from both work indicate that zeroshot and few-shot prompting of llms continues to significantly underperform when compared to the results obtained through fine-tuning and prompttuning plms. the same outcome has been observed in other idrr work focusing on different languages @xcite . the recent move toward multi-label annotation in discourse data has led to initial efforts in multilabel idrr. @xcite used the 4.9% of implicit discourse relations in the pdtb 3.0 corpus that are annotated with two senses to build a model capable of predicting up to two senses per instance. however, given that most of pdtb 3.0 annotations remain single-label, their model predominantly produces single-label predictions. in contrast, @xcite and costa and kosseim (2024) used the multi-label discogem 1.0 corpus but convert its annotations into a single-label format during training. finally, costa and kosseim (2025) present the first results in multi-label idrr by training a model on discogem 1.0 that jointly predicts sense distributions across all three levels in the pdtb 3.0 framework. rather than adapting multi-label data to a single-label setting or modifying a single-label corpus for multi-label use, they preserve the inherent multi-label structure of disco-gem 1.0 throughout training. however, like most previous work in idrr, their model is limited to the english language. ## our approach we propose a hierarchical multi-task classification model, harch, to predict probability distributions across all three sense levels in the pdtb 3.0 framework. we also propose a prompting framework to compare our harch model against sota llms through few-shot learning. ## data preparation to train our model on multi-lingual idrr, we use the recently released discogem 2.0 corpus @xcite . the corpus comprises a total of 13,063 foot_0 parallel annotated implicit discourse relations across four languages: 5,847 in english, 2,588 in german, 2,628 in french, and 2,000 in czech. all annotations follow the pdtb 3.0 discourse framework @xcite . while its predecessor, discogem 1.0 @xcite , provided the first corpus of multi-label implicit discourse relations, discogem 2.0 is the first corpus to provide multi-lingual parallel annotation of multi-label implicit discourse relations. each implicit discourse relation in the corpus was annotated by at least 10 crowdworkers. each annotator selected the sense they considered most appropriate for each relation through a proxy task and then all of the annotated senses of each relation were averaged to produce a multi-label sense distribution over 28 different senses in the pdtb 3.0 framework -the belief and the speech-act senses were not annotated. for each language, the authors of the corpus prepared a list of discourse connectives (one representing each of the available senses) and asked the annotators to select for each relation the discourse connective that best expresses the semantic relation between its two arguments. from the chosen connectives, the authors were able to infer the possible multiple senses of each implicit discourse relation. to ensure reproducibility of our results, we split the discogem 2.0 corpus following the proposed training, validation and test splits @xcite . contrary to the common approach in traditional single-label idrr, we follow the methodology of costa and kosseim (2025) and consider all three sense levels in the pdtb 3.0 framework when possible. ## encoder selection in this section, we compare the performance of different plms as encoders in our harch model (see figure 1 ) for multi-label idrr in both multilingual and english-only settings, using the disco-gem 2.0 corpus. next, we explored the impact of using different plms as encoder backbones for our harch hierarchical model. in particular, we experimented with modernbert base @xcite , a recently introduced encoder claimed to outperform roberta, and the multi-lingual xlm-roberta base @xcite . contrary to expectations, modernbert-harch consistently underperformed across all three sense levels, as shown in ## results in this section, we report the performance of our harch models on the test splits of both disco-gem 1.0 (see we report results on discogem 1.0 to benchmark our approach with the only other existing model for multi-label idrr that predicts probability distributions over the full set of pdtb 3.0 sense labels (costa and kosseim, 2025). since discogem 1.0 contains only english data, we report the performance of our best-performing english model in an ablation study by training three individual models -each dedicated to predicting probability distributions for a single sense level. this study was conducted in the english and multi-lingual setting. to ensure a fair comparison, each individual model tries to replicate the architecture of the multi-task model (see figure 1 ), with the exception that it includes only one classification head and no augmentation blocks. specifically, each individual model encodes a concatenated pair of discourse arguments using the corresponding plm, which it then passes to a linear transformation and dropout layer, before feeding the representation to a classification head -similar to the level-1 prediction path in the multi-task model. as shown in ## conclusion in this work, we presented the first multi-lingual and multi-label classification model for idrr. we modeled hierarchical dependencies between discourse senses to predict probability distributions across all three sense levels in the pdtb 3.0 framework. we introduced the first evaluation on the discogem 2.0 corpus across all languages and compared different pre-trained language models as encoder backbones within our proposed harch model architecture. results show that roberta-harch achieves the best performance in english, while xlm-roberta-harch performs best in the multi-lingual setting. we further evaluated our models against gpt-4o and llama-4-maverick using few-shot prompting across all language configurations and found that the harch models consistently obtained lower js distances. these findings demonstrate that fine-tuning smaller encoder-based models can outperform prompting large language models in the context of multi-label idrr. lastly, we benchmarked roberta-harch on the englishonly discogem 1.0 corpus and achieved sota results compared to existing work. these findings not only advance the field of idrr, but also highlight the advantages of finetuning smaller encoder-based models over prompting large language models -particularly, given the significantly higher monetary and environmental costs associated with the latter. looking ahead, we plan to investigate how discourse sense predictions vary across languages in discogem 2.0 for parallel discourse relations. in particular, we are interested in exploring how translation influences discourse reasoning and label distribution across these four languages. additionally, we aim to compare the performance of using xlm-roberta as the encoder in our harch model in the german, french and czech languages against encoders pretrained specifically in these languages. this line of research may shed light on cross-linguistic challenges in discourse analysis and lead to more robust multi-lingual idrr models.