A Topicality-Driven QUD Model for Discourse Processing (2025)

Abstract: Question Under Discussion (QUD) is a discourse framework that has attracted growing interest in NLP in recent years. Among existing QUD models, the QUD tree approach (Riester, 2019) focuses on reconstructing QUDs and their hierarchical relationships, using a single tree to represent discourse structure. Prior implementation shows moderate inter-annotator agreement, highlighting the challenging nature of this task. In this paper, we propose a new QUD model for annotating hierarchical discourse structure. Our annotation achieves high inter-annotator agreement: 81.45% for short files and 79.53% for long files of Wall Street Journal articles. We show preliminary results on using GPT-4 for automatic annotation, which suggests that one of the best-performing LLMs still struggles with capturing hierarchical discourse structure. Moreover, we compare the annotations with RST annotations. Lastly, we present an approach for integrating hierarchical and local discourse relation annotations with the proposed model.

## introduction as large language models (llms) show impressive performance on various nlp tasks @xcite @xcite , there is an increasing number of studies that try to convert nlp tasks into text generation tasks to leverage the strong generative capability of llms @xcite . in computational discourse processing, question under discussion (qud) @xcite @xcite has been attracting growing attention over the years. with this framework, discourse units, typically sentences, are considered as answers to some explicit or implicit questions, which are called quds. one qud may lead to 1. of all the ethnic tensions in america, which is the most troublesome right now? 2. a good bet would be the tension between blacks and jews in new york city. ## related work there exists a substantial body of work on discourse modelling. rst, pdtb, and segmented discourse representation theory (sdrt) @xcite are the canonical discourse frameworks. we provide a review of related discourse models to clarify the conceptualization and motivations of our research. qud-tree this is the most closely related work to the present study. it can be traced to the model by @xcite . @xcite argues that people communicate to reach a common ground and quds are raised during the process of communication to resolve indeterminacies. when a qud is accepted, it is put on a stack model of discourse, and discourse participants are committed to answering it. it is popped off the stack when it is answered satisfactorily or when it is determined to be unanswerable. accordingly, discourse can be modeled by a single tree of quds. @xcite develops four constraints for reconstructing quds. these constraints also form the theoretical foundation for later studies on evaluating qud parsing @xcite . despite the common assumption of using a single tree in modelling discourse, @xcite and de @xcite depart from the thesis of @xcite and allow a weaker connection between quds than the criteria defined by @xcite : a qud is considered valid as long as it is topically related to the qud at the top of the stack. for discourse segmentation, information-structural units are used as basic units, which can be clauses, and in many cases, they are more fine-grained. rst rst is the closest related mainstream discourse framework to the qud-tree approach. similar to qud-tree, the first step for rst analysis is typically discourse segmentation. clauses are the basic discourse units, called elementary discourse units (edus) @xcite . to improve consistency in this step, rules based on syntactic clues are introduced in the creation of the rst discourse treebank (rst-dt) @xcite , which is built on wall street journal (wsj) articles used in the penn treebank @xcite . edus are linked through rhetorical relations and discourse units are grouped in this way recursively until a tree is formed. the relationship between rst and qud-tree has been investigated in some studies @xcite . topic segmentation topic segmentation refers to the task of breaking down a text into groupings of smaller segments, where the motivation for the grouping is that the segments deal with a common topic @xcite . most existing studies on topic segmentation focus on sentence-level topic boundary demarcation @xcite . in comparison, @xcite take paragraphs as the basic units under the assumption that each paragraph deals with one topic. consecutive paragraphs may be grouped if they address the same topic, with the document as a whole focusing on a supertopic. thus, the structure is hierarchical. however, in @xcite 's study, paragraphs dealing with the same topic are simply grouped, without considering the relationship between paragraphs under the same topic or between topics, and they work with coarse-grained textual units. ## theoretical framework in the theoretical model proposed by @xcite , the process of questioning involves three parameters: feeders, topic-constituting questions and subtopic-constituting subquestions (henceforth referred to as subquestions for brevity). feeders are discourse units that trigger a questioning process, and they serve to introduce indeterminacies, which questions are then raised to resolve. feeders can be linguistic, such as the opening sentence of a monologue, or non-linguistic, for instance, a door knock that triggers a conversation. the defining characteristic of a feeder is that the discourse unit is either topic-less, not prominent for the current discourse, or occurs in a place of the text where no context is set for it. in the example given in the lower part of figure 1 (henceforth "london underground example"), the first sentence functions as a feeder because no context is set for it, and accordingly, no quds are reconstructed for this discourse unit. following @xcite , @xcite proposes to add a proxy qud for feeders -"what is the way things are?". another example is provided below to illustrate different types of feeders (a modified version of example (10) from van @xcite , henceforth "mary's holiday example"). there are two feeders, which initiate different discourse topics. in this dialogue, the first feeder is of the same type as that in the london underground example, while the second feeder belongs to a different type, which originates from a part of the previous topic, i.e.,"going to thailand". the introduction of a feeder f initiates a discourse topic, dt , which is typically unfolded as a set of topic-constituting questions {t q 1 , t q 2 , t q 3 , . . .}. hence, a discourse topic overlaps with a topic-constituting question when only one topic-constituting question is induced. as indicated by van kuppevelt (1995), topicconstituting questions have an autonomous status in discourse. these higher-order questions impose a restriction on the development of discourse. an unsatisfactory answer to such questions gives rise to a subquestion, which, if not answered satisfactorily, brings up a further subquestion, and so on, until no further subquestions are elicited, leading to a hierarchical structure of questions. therefore, different from topic-constituting questions, subquestions are contextually induced and do not have an autonomous status in discourse. as shown by the london underground example, q1 and q3 are topic-constituting questions, one concerning people's reaction and the other dealing with the consequence of the conclusion. q2 is induced from a1, which suggests that a1 provides an unsatisfactory answer to q1, and in this case, q2 forms a subquestion of q1. the process of pursuing a topic-constituting question by raising subquestions is governed by two principles: the principle of recency and the dynamic principle of topic termination. the principle of recency suggests that subquestions are evoked by the most recent unsatisfactory answer to a preceding question. in the london underground example, the subquestion q2 is induced by a1, which is the most recent unsatisfactory answer to q1. this principle ultimately produces a tree for a topic-constituting question, akin to the stack model proposed by @xcite . the dynamic principle of topic termination specifies that if an explicit or implicit question is answered satisfactorily, the related questioning process is closed, and the topic carried by the question loses its currency. this explains why in the mary's holiday example, when a2 is given, the topic associated with q2 loses its relevance and the next sentence functions as a feeder. a method can be used to determine if the question of a discourse unit is a subquestion of the immediately preceding question: a topic-closing sentence s is added after an answer a i to a question q i . if the immediately following question q i+1 after a i becomes infelicitous, it means that q i+1 is a subquestion of q i . this method is called subordination test. an example of s is "i now understand that...", which forces a preceding topic to be closed. in the london underground example, when a subordination test is applied to test if q3 is subordinate to q2: f: yesterday a jury of investigation came to the conclusion that... q1: how did people react to the outcome of the investigation? a1: relatives of the victims rejected it. q2: why? a2: they are of the opinion that the jury did not do their job well. s: i now understand why relatives of the victims rejected it. q3: what is the consequence of the conclusion? a3: further prosecution of the officials of london regional transport is ruled out. q3 is felicitous given s. therefore, q3 is not subordinate to q2. in the model by @xcite , a qud that has been answered and popped off the stack is added to the common ground of the discourse model, contributing to a cumulative and evolving common ground that further shapes the development of discourse. in contrast, with the model by van @xcite , questions that have been answered satisfactorily lose their actuality, indicating a transient and dynamic process. moreover, the model by @xcite assumes one primary communicative goal-to come to an agreement about the way things are. thus, discourse is organized around accumulating shared knowledge and reaching consensus. consequently, discourse is represented by a single hierarchical tree. by comparison, with the model by van @xcite , a discourse topic is defined by a set of topic-constituting questions that form a program about how discourse progresses. it is conceivable that the resulting structure consists of a set of shallower trees. apart from discourse structuring, discourse segmentation is also shaped by the organization of questions. with the model, multiple discourse topics can be formed, which is typical for informal conversations, and these topics can be structurally unrelated. at the top level, discourse units of larger sizes address discourse topics. beneath each discourse topic unit, one or more smaller discourse units can be identified, each associated with topicconstituting questions linked by the common discourse topic. furthermore, each discourse unit answering topic-constituting questions may be further divided into finer-grained units based on the subquestions they address. the dynamic principle of topic termination entails topic shifts, which may occur when one topicconstituting question is resolved and discourse progresses to the next topic-constituting question. as mentioned earlier, a feeder initiates a discourse topic. therefore, two types of topic shifts may be defined: topic shifts under the same feeder (one discourse topic), and topic shifts under successive feeders (more than one discourse topic). topic shifts under the same feeder happen when a new topic-constituting question arises. the case of topic shifts under successive feeders is more complicated, involving three possibilities: associated topic shifts, non-associated topic shifts and topic descending shifts. associated topic shifts refer to the case when the new feeder is contained within a part of the preceding discourse. in the mary's holiday example, f2 is associated with a2 in the previous discourse. in comparison, non-associated topic shifts occur when the new feeder is irrelevant to the preceding discourse. this is typical in informal conversations, where participants frequently jump from one topic to another. topic descending shifts form a special case of associated topic shifts, where a preceding subtopic under an old discourse topic becomes the feeder for the new discourse topic, as shown by the following example (originally from van kuppevelt (1995), edited): hence, the model allows for capturing incoherence caused by topic shifts, which are common in dialogues. ## comparison with rst trees owing to the similarity in encoding hierarchical discourse structure, we make a comparison between the annotated qud trees and rst trees. as sentences are used as the basic unit in the present study, only rst trees above the sentence level are extracted. when assuming a discourse topic at the top, the structure produced by the qud model may be closer to rst. therefore, when comparing the heights of rst and qud trees, the heights of qud trees are computed under the assumption that an overall discourse topic is present. even though there are spans that are annotated with an rst relation and a qud in parallel, it is difficult to draw any conclusions on the relationship between rst relations and quds, because questions can be formulated in different ways. for example, as shown in ## integrating local and hierarchical annotations most existing studies on qud treat sentences as the basic unit @xcite ). an exception is the research by @xcite , which allows for the inclusion of sub-sentential units. however, a significant proportion of these units do not possess their own quds, leading to cases of quds with joint answers (appendix h). since sentences are the basic discourse units, the current qud model does not consider intra-sentential relations. one reason is that it is often challenging to identify topics for sub-sentential units. following the line of research on integrating different perspectives of discourse annotation @xcite @xcite @xcite , we show a method of integrating local and hierarchical discourse annotations to enable a full question answering (qa) approach for discourse processing, with the proposed model for processing inter-sentential relations and the method proposed by @xcite for handling intra-sentential relations. the method by @xcite has been shown to be applicable to naturally occurring texts. however, their experiments involve a preprocessing step of target extraction, which might introduce biases for our task, as discourse relations are not necessarily inferred based on the relationships between entities. therefore, we adopt the criteria of determining intra-sentential arguments in pdtb 3.0 @xcite as a first step to identify discourse units at the intra-sentential level. similar to @xcite , we allow more than one question to be raised for a pair of intra-sentential arguments. therefore, given a list of sentences, intra-sentential level processing involves two subtasks: identifying the argument pairs within each sentence (more than one pair may be possible), and converting these argument pairs into qa pairs using the question templates provided by @xcite . ## conclusions we introduce and implement the topicality-driven qud model proposed by van kuppevelt (1995) for annotating hierarchical discourse structure. we position the model in existing discourse frameworks and analyze its properties. with this model, high iaa can be achieved on challenging, naturally occurring texts. we compare the annotations with rst annotations to have a better understanding of the generated hierarchical structures and the correlation between quds and rst relations. since this model takes sentences as the basic discourse unit, the method proposed by @xcite is adopted for intra-sentential level processing, thereby enabling a full qa approach for discourse processing.