D isco F lan: Instruction Fine-tuning and Refined Text Generation for Discourse Relation Label Classification (2023)

Abstract: This paper introduces DiscoFlan, a multilingual discourse relation classifier submitted for DISRPT 2023. Our submission represents the first attempt at building a multilingual discourse relation classifier for the DISRPT 2023 shared task. By our model addresses the issue to mismatches caused by hallucination in a seq2seq model by utilizing the label distribution information for label generation. In contrast to the previous state-of-the-art model, our approach eliminates the need for hand-crafted features in computing the discourse relation classes. Furthermore, we propose a novel label generation mechanism that anchors the labels to a fixed set by selectively enhancing training on the decoder model. Our experimental results demonstrate that our model surpasses the current state-of-the-art performance in 11 out of the 26 datasets considered, however the submitted model compatible with provided evaluation scripts is better in 7 out of 26 considered datasets, while demonstrating competitive results in the rest. Overall, DiscoFlan showcases promising advancements in multilingual discourse relation classification for the DISRPT 2023 shared task.

## introduction discourse relation classification (drc) is a discourse-level task that requires the identification of discourse relations between text segments in a document. this low-resourced task contains multiple subtasks with different languages and formalisms. the numbers of unique labels vary from 9 in zho.pdtb.cdtb to 33 in nld.rst.nldt. we train discoflan and compare it with the current state-of-the-art model as well as a multilingual classification baseline on the disrpt datasets and present our results for the 2023 disrpt sharedtask for discourse relation classification. supervised large language models (llms) trained with human labels are truly a paradigm shift due to their zero-shot and low-resource capabilities. improved language representation mechanisms and utilization of large pre-training corpora of llms have led to significant advancements in two key areas: zero-shot capabilities and lowresource prompt learning. in this paper, we focus on low-resource drc. such breakthroughs are a testament to the power of large-scale pretraining. with enhanced representations, language models can generalize and transfer knowledge across different tasks and domains, enabling impressive zeroshot capabilities where models can perform well on tasks they were not explicitly trained on. these improved zero-shot learners have the capability to learn efficiently on low-resource complex tasks like relation classification. the availability of even limited amounts of training data allows for effective low-resource prompt learning. these advancements highlight the immense potential of llms and their ability to tackle the real-world problem of drc. our main contributions are: 1. we perform instruction finetuning of multilingual prompts with discoflan for drc tasks of different formalisms and languages to develop a seq2seq generative label classification system. 2. we use a simple postprocessing stage harnessing the label distributions using majority label distributions for low resource dataset. foot_0 2 related work