Clarifying Underspecified Discourse Relations in Instructional Texts (2025)

Abstract: Discourse relations contribute to the structure of a text and can optionally be realized through explicit connectives such as “but” and “while”. But when are these connectives necessary to avoid possible misunderstandings? We investigate this question by first building a corpus of 4,274 text revisions in each of which a connective was explicitly inserted. For a subset of 250 cases, we collect plausibility annotations on other connectives to check whether they would represent suitable alternative relations. The results of this annotation show that several relations are often perceived as plausible in our data. Furthermore, we analyze the extent to which large language models can identify instances with multiple plausible relations as a possible source of misunderstandings. We find that the models predict plausibility of individual connectives with up to 66% accuracy, but they are not reliable in estimating when multiple relations are plausible.

## introduction discourse relations play a crucial role in establishing coherence and logical flow between discourse segments in natural language. unlike genres such as narratives or news, where ambiguity may be acceptable or even desirable, instructional texts should have a single clear meaning to ensure that readers can effectively follow the instructions. that is, while discourse relations in general can often be inferred by readers implicitly based on context and prior knowledge, instructions for a new task or unfamiliar domain may require the connection between steps to be explicit to avoid confusion (e.g. when steps can be carried out in any order versus only in a specific sequence). however, it has not been fully explored when the introduction of a connective (e.g. 'meanwhile', 'afterwards') alters the perceived plausibility of discourse relations. as a starting point for studying when an implicit or explicit relation affects plausibility, we how to become a registered nurse (...) obtain a bachelor's degree in nursing. * programs typically take four years to complete, and vary in cost depending on which institution you choose. ∅ bachelor's programs usually include more training in social sciences than other nursing programs. ## background as a general framework for (shallow) discourse relations, we rely on the penn discourse treebank (pdtb), which we discuss in more detail below ( §2.1). we further motivate this choice over other frameworks in section 2.2. ## related work explicitation of discourse relations has been widely studied in the context of translation studies, particularly related to the explicitation hypothesis, which suggests that translations tend to be more explicit than their source texts (blum-kulka, 1986). numerous studies have examined parallel texts to explore this hypothesis (e.g., zufferey and cartoni, 2014; crible et al., 2019; lapshinova-koltunski et al., 2022), focusing largely on the insertion or omission of the connective in the translations. yung et al. (2023) explore another form of explicitation, where a more specific connective is used in translation (e.g., translating "and" as "außerdem" in german), providing further evidence for the explicitation hypothesis. in a study related to ours, rohde et al. (2016) examine the interpretation of discourse relations and, through a crowdsourcing study framed as a connective insertion task, show that explicit markers and inferred conjunctions can coexist. this challenges the assumption that discourse relations are either explicit or inferred. in another crowdsourcing study, yung et al. (2019) introduce a two-step method where workers first insert and then disambiguate connectives to annotate discourse relations, a method used for the discogem corpus (scholman et al., 2022). yung et al. (2024b) later refine this into a one-step procedure for annotating the discogem 2.0 corpus across multiple languages. in the pdtb, the annotation of implicit relations involves first inserting a connective between the arguments, followed by labeling the relation's sense (prasad et al., 2008). building on this approach, several works show that generating discourse connectives between the arguments of implicit relations enhances classification (shi and demberg, 2019; zhou et al., 2022; xiang et al., 2022; liu and strube, 2023; wang et al., 2023; wu et al., 2023). our dataset, which includes numerous instances of connective insertion, can support future research on the automatic recognition of implicit relations, in particular with regard to settings where multiple relations are plausible. ## data collection the goal of this work is to analyze to what extent implicit discourse relations may require clarification in order to avoid potential of misunderstandings. as a foundation for this analysis, we construct a data set of discourse relation explicitations through connective insertion and plausibility judgments. we proceed in three steps: first, we extract 4,274 instances of connective insertions from wikihow by comparing different versions of the same article ( §4.1); second, we generate alternative connectives indicating different relations that may also be plausible ( §4.2); and third, we collect annotations from human judges on which individual connectives are perceived as plausible ( §4.3). finally, we provide statistics on the dataset ( §4.4). ## data analysis a fundamental question of our work is whether the connective insertions observed in the data are necessary to avoid possible causes of misunderstandings. to answer this question, we examine how frequently different connectives in the same context are perceived as plausible. as shown in ## computational experiments in section 4, we discussed the creation of our data and showed that multiple plausible relations exist for many of the selected contexts (81%). our analysis in section 5 has shown that some of the relations perceived as plausible are mutually exclusive, indicating that a lack of discourse relation explicitation could lead to misunderstanding. in the following, we investigate whether more recent large language models (llms) can predict which connectives are plausible in a given context @xcite and, by extension, whether multiple relations are plausible, signalling a potential source of misunderstanding. ## discussion we briefly summarize our findings with respect to the research questions introduced in section 1. rq1. how frequently are multiple relations perceived as plausible for the same context in our data? gold annotation counts by humans are shown above model prediction counts for each class and corresponding results. as evaluation measures, we report precision (p), recall (r), and their harmonic mean (f 1 ) per class, along with weighted f 1 -score (wf 1 ) and overall accuracy (acc.) across classes. on the other hand, how commonly is insertion of a connective redundant or unnecessary? for the vast majority of the arguments, we found in the collected data that two or more connectives are annotated as plausible (81%, see §4.4). in contrast, there are only a few cases where the original connective is perceived as the only plausible option (7%). this indicates that the insertion of connectives as part of a revision is usually not redundant. ## conclusion we introduced a dataset of 4,274 text revisions with explicit connective insertions, along with a subset containing crowdsourced plausibility judgments on alternative connectives. in many cases, we found that not only was the inserted connective perceived as plausible, but other alternative connectives, sometimes indicating different discourse relations, were also judged as plausible. for relations that are mutually exclusive, this suggests that the lack of an explicit connective oftentimes implies a potential source of misunderstanding. we also conducted experiments with llms to automatically predict the plausibility of different markers in the same contexts. while the results are promising regarding individual relations, llms struggle to correctly identify when multiple or conflicting relations are plausible. future research should investigate in more depth the extent to which models exhibit discourse understanding relevant to our task. for example, miao et al. (2024) propose a question-answering-based method to evaluate the "faithfulness" by testing whether llms predictions remain consistent when the direction of a discourse relation between the same arguments is reversed (e.g., result vs. reason). while our study is grounded in the pdtb framework, the focus on misunderstandings that can arise from missing explicit connectives suggests that applying a dynamic and structured framework such as sdrt could offer further valuable insights.