A Multi-Task and Multi-Label Classification Model for Implicit Discourse Relation Recognition (2025)

Abstract: We propose a novel multi-label classification approach to implicit discourse relation recognition (IDRR). Our approach features a multi-task model that jointly learns multi-label representations of implicit discourse relations across all three sense levels in the PDTB 3.0 framework. The model can also be adapted to the traditional single-label IDRR setting by selecting the sense with the highest probability in the multi-label representation. We conduct extensive experiments to identify optimal model configurations and loss functions in both settings. Our approach establishes the first benchmark for multi-label IDRR and achieves SOTA results on single-label IDRR using DiscoGeM. Finally, we evaluate our model on the PDTB 3.0 corpus in the single-label setting, presenting the first analysis of transfer learning between the DiscoGeM and PDTB 3.0 corpora for IDRR.

## introduction implicit discourse relation recognition (idrr) is one of the most challenging tasks in computational discourse analysis. its goal is to identify the sense of discourse relations connecting text arguments in the absence of explicit discourse connectives, such as but and because. the penn discourse treebank (pdtb) @xcite organizes discourse senses across three hierarchical levels with increasing degrees of detail @xcite . however, understanding the sense of a discourse relation can be a complex and subjective task without the guidance of discourse connectives. consider the two arguments of the implicit discourse relation below: arg1: the lights flickered. arg2: the power went out. in this example, it is not clear whether the two arguments of the relation are related by a temporal or causality relation. one could interpret that the lights flicked and then the power went out, or that the lights flickered because the power went out. this ambiguity illustrates the biggest challenge of idrr. despite its difficulty, idrr plays a crucial role in downstream tasks that rely on text coherence. in summarization, for example, it helps preserve the logical flow and rhetorical intent of the source text. in dialogue systems, it supports coherence and intent modeling by inferring how utterances relate to each other, improving dialogue discourse parsing @xcite . to date, most research on idrr has relied on the different iterations of the pdtb corpus @xcite . however, despite significant efforts, sota performance on idrr has plateaued in recent years at f1-scores of 71.59 at level-1 and 57.62 at level-2 @xcite @xcite @xcite . one possible reason may be the inherently subjective nature of implicit discourse interpretation, which can be difficult to capture using predominantly single-label annotated corpora @xcite @xcite . the idea of possible multiple relations between discourse arguments had already been considered in the segmented discourse representation theory (sdrt) @xcite and the ambiguity in the annotation of implicit relations has been further highlighted by the challenges in mapping them across discourse frameworks @xcite . in light of these challenges, recent studies have advocated for the multi-label annotation of implicit relations to better capture their nuanced and complex nature in discourse corpora @xcite @xcite @xcite . this shift in perspective has led to the creation of discogem @xcite ) -the first multi-label annotated corpus of implicit discourse relations. following recent work in nlp @xcite @xcite @xcite @xcite , we see disagreement between humans annotators not as noise, but as an opportunity to enhance the relevance of idrr models by capturing the complexity of human interpretations. this paper addresses the subjectivity in idrr by proposing a multi-task classification model capable of learning both multi-label and single-label representations of implicit discourse relations. our model is trained on the discogem corpus and uses a multitask architecture to leverage the interdependence between senses and jointly learn sense representations across all three sense levels in the pdtb 3.0 hierarchy. the main contributions of this work are: our multi-label model can then be adapted to the traditional single-label task by selecting the label with the highest probability. ## previous work despite the recent advances in natural language understanding, idrr remains one of the most challenging tasks in discourse analysis. most previous work addressed idrr as a single-label classification task by fine-tuning @xcite or prompt-tuning @xcite @xcite pre-trained language models (plms). more precisely, long and webber (2022) apply contrastive learning and augment training data with explicit connectives from pdtb 3.0 metadata to fine-tune their model, while @xcite propose a two-step pipeline that first generates an explicit connective for each relation to fine-tune a classifier on the modified input. @xcite inject hierarchical structure and connective-based explanations into prompts, enabling joint predictions across all three pdtb sense levels. @xcite address data scarcity in idrr by using a parameter-efficient prompt-tuning framework that incorporates hierarchical label guidance into the verbalizer. building on this, @xcite propose a prompttuning approach that integrates both global and local hierarchical label information into the verbalizer to improve output alignment with pre-trained objectives. finally, @xcite introduce a prototype-based verbalizer informed by the pdtb 3.0 sense hierarchy, combining contrastive and prototype learning to eliminate the need for manually designed verbalizers. while most of these approaches @xcite @xcite @xcite use roberta base @xcite as their plm, @xcite uses t5 base @xcite . another line of work tried to solve single-label idrr by directly prompting large language models (llms) through promptengineering @xcite . however, both works show that the results obtained through directly prompting llms in zero-shot and few-shot settings are still far behind from those obtained through fine-tuning and prompt-tuning plms. the shift toward multi-label annotation has stirred an initial wave of research in multi-label idrr. for instance, @xcite used the 4.9% of implicit relations in the pdtb 3.0 corpus that are annotated with two senses to build a model capable of predicting up to two senses per instance. however, given that the 95.1% of pdtb 3.0 annotations remain single-label, their model predominantly produces single-label predictions. in contrast, @xcite and costa and kosseim (2024) also use the multi-label discogem corpus but convert its annotations into a singlelabel format during training. thus, to date, no previous work has truly captured the full multilabel representation of implicit discourse relations -with the exception of our work on multi-lingual idrr (costa and kosseim, 2025), where we consider multi-label idrr in a multi-lingual setting incorporating hierarchical learning in the training of our classification model and comparing it against llms via direct prompting with few-shot learning. similar to other works @xcite , we found that prompting llms leads to worst results compared to the fine-tuning of plms such as roberta in the context of idrr. therefore, we do not consider prompting llms in this paper. ## our approach in this work, we introduce a novel multi-task classification model for idrr that simultaneously predicts sense distributions across all three levels in the pdtb 3.0 framework. as illustrated in figure 1 , the model processes a concatenated pair of discourse arguments (arg1+arg2) using roberta @xcite as the plm encoder 1 . the resulting contextualized embedding is passed through a linear transformation and dropout layer and subsequently fed to three distinct classification heads -each corresponding to a sense level in the hierarchy. in the multi-label setting, each head outputs a probability distribution over the available senses at its respective level. in the single-label setting, we apply an additional pooling layer that selects the sense with the highest probability from each distribution. we use the adam optimizer @xcite to minimize the loss function, which we calculate as the sum of the individual losses of each classification head. multi-label classification. for each multi-label classification head, we compute the loss using the mean absolute error (mae) loss function (see equation 2 in appendix b), which we found to yield better performance (as detailed in section 5.1). following previous work in multi-label classification for nlp @xcite , we evaluate model performance in this setting using the jensen-shannon (js) distance @xcite to measure the similarity between the predicted and target probability distributions. single-label classification. in the single-label setting, we compute the loss for each classification head using the cross-entropy (ce) loss function (see equation 1 in appendix b), which emphasizes the correct classification of the highest probability sense label (as discussed in section 5.1). we evaluate model performance in this setting using the 1 we also experimented with other plms, but roberta achieved the best performance (see section 5.1). weighted f1-score based on the majority label in the predicted and target sense distributions. ## data preparation we used two different discourse annotated corpora in this work: discogem @xcite and pdtb 3.0 @xcite . our model was trained exclusively on the discogem corpus and evaluated on both discogem (for multi-label and single-label classification) and on pdtb 3.0 (for single-label classification). although the two corpora differ in annotation methodologies, both follow the same discourse framework @xcite . this shared foundation allowed us to directly evaluate the transfer learning potential of training our model on discogem and evaluating it on pdtb 3.0 in single-label iddr. ## experiments to optimize our model (illustrated in figure 1 ), we explored multiple model configurations and loss functions using the validation set of discogem. all of the results reported in the following sections are the average scores of three different runs with different random starts for 10 epochs and with a batch size of 16. we ran our experiments on a 32core compute node with 512gb of ram. all of the code used can be found on github foot_0 . ## results in this section, we present the results of our experiments in two parts. section 6.1 reports the performance of our models on the discogem test set for both multi-label and single-label idrr and section 6.2 presents the transfer learning results of our single-label model evaluated on various test splits of the pdtb 3.0 corpus. ## analysis to further evaluate the performance of our model, we conduct two additional analyses. in section 7.1, we perform a per-sense evaluation to assess the ability of the model to correctly classify individual senses at both level-1 and level-2. in section 7.2, we evaluate the cross-level coherence of the predictions by examining how well the predicted labels at level-1 and level-2 align with the sense hierarchy defined in the pdtb 3.0 framework. for ease of in- ## conclusion in this work, we proposed a novel multi-label framework for idrr, addressing one of the most complex challenges in discourse analysis. we trained a multi-task classification model on the discogem corpus to simultaneously learn multilabel representations of discourse relations across all three sense levels in the pdtb 3.0 framework. our model can also be adapted to the traditional single-label idrr setting by selecting the sense with the highest probability in the multi-label representation. we conducted extensive experiments to identify optimal model configurations and loss functions in both settings. our approach establishes a first benchmark on multi-label idrr and achieves sota results in single-label idrr using discogem. finally, we evaluated our model on the pdtb 3.0 corpus in the single-label setting, presenting the first analysis of transfer learning between the discogem and pdtb 3.0 corpora for idrr. our results show that zero-shot direct transfer learning between both corpora still needs further research. future work should explore the application of data augmentation techniques, such as paraphrasing, to determine whether augmenting underrepresented senses in the discogem could enhance the performance of the model on individual senses. additionally, future research should investigate whether cascading information from higherlevel to lower-level classification heads within the model could improve the coherence of its predictions across sense levels. lastly, it would be worthwhile to examine whether a model trained to learn multi-label representations of discourse relations in discogem could be further fine-tuned on the pdtb 3.0 to achieve superior performance in single-label idrr using the pdtb 3.0 corpus.