Let’s be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction (2021)

Abstract: In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. This is challenging even for humans, leading to shortage of annotated data, a fact that makes the task even more difficult for supervised machine learning approaches. In the current study, we perform implicit discourse relation classification without relying on any labeled implicit relation. We sidestep the lack of data through explicitation of implicit relations to reduce the task to two sub-problems: language modeling and explicit discourse relation classification, a much easier problem. Our experimental results show that this method can even marginally outperform the state-of-the-art, in spite of being much simpler than alternative models of comparable performance. Moreover, we show that the achieved performance is robust across domains as suggested by the zero-shot experiments on a completely different domain. This indicates that recent advances in language modeling have made language models sufficiently good at capturing inter-sentence relations without the help of explicit discourse markers.

## introduction discourse relations describe the relationship between discourse units, e.g. clauses or sentences. these relations are either signalled explicitly with a discourse connective (e.g. because, and) or expressed implicitly and are inferred by sequential reading (example 1 below). ## model the proposed method consists of three main components: (i) a candidate generator that generates sentence pairs connected by each of a set of discourse connectives, (ii) a language model that estimates the likelihood of each candidate, and (iii) an explicit discourse relation classifier to be used on the candidates. whole pipeline is shown in figure 1. the proposed methodology does not require even a single implicit discourse relation annotation and is only distantly supervised where the supervision comes from the explicit discourse relations used in training the classifier. the main motivation behind the proposed pipeline is the finding that discourse relations are easily classifiable if they are explicitly marked @xcite . we further verify this finding via a preliminary experiment which showed that four-way classification could be performed with an f-score of 88.74 when the implicit discourse relations are "explicitated" with the gold implicit connectives they are annotated with (see ## experiments we follow the experimental setting of @xcite which is originally adopted by @xcite . the implicit relations in the pdtb 2.0 sections 21-22 are allocated as the test set whereas the explicit relations in sections 2-20;23-24 are used as the training and 0-1 as the development set of the explicit relation classifier. the evaluation is performed for both the four first-level and the most common 11 second-level senses. for the former, we report both per-class and the macro-average f1-scores similar to @xcite whereas the accuracy is also reported on the second level the classifiers are implemented using the transformers library by huggingface @xcite . we use the uncased bert large model for the explicit relation classifier (section 3.3). the model is fine-tuned for ten epochs with a batch size of 16, learning rate of 5 × 10 -6 . to optimize the loss function, we use adam with fixed weight decay @xcite and warm-up linearly for the first 1k steps. the model is evaluated with the step size of 500 and the one with the best development performance is used as the final model. we mainly compare our results against the recent unsupervised studies we are aware of @xcite . additionally, we report the performance of a number baselines and upper bounds to put the results into a perspective: ## conclusions in addition to its inherent difficulty, implicit discourse relation classification becomes even more challenging with the lack of sufficient data. in the current study, we focus on the latter problem by assuming the extreme low-resource scenario where there are no labeled implicit discourse relations. the data shortage is mitigated by leveraging the contextual information of the available pre-trained language models through explicitation of the implicit relations. we show that the proposed pipeline, despite its simplicity, is able to outperform the previous attempts. furthermore, by taking another step, we tested the proposed architecture in the more challenging 11-way setting as well as on a completely different domain. the experimental results confirm that our model is robust and generalizes well, even compared to recent supervised approaches.