The RST Continuity Corpus (2023)

Abstract: We present the RST Continuity Corpus (RST-CC), a corpus of discourse relations annotated for continuity dimensions. Continuity or discontinuity (maintaining or shifting deictic centres across discourse segments) is an important property of discourse relations, but the two are correlated in greatly varying ways. To analyse this correlation, the relations in the RST-CC are annotated using operationalised versions of Givón’s (1993) continuity dimensions. We also report on the inter-annotator agreement, and discuss recurrent annotation issues. First results show substantial variation of continuity dimensions within and across relation types.

## introduction in this paper, we present the rst continuity corpus (rst-cc), the first corpus of discourse relations annotated for a wide range of continuity dimensions (e.g., time, space, reference, or perspective). these dimensions describe different ways in which a deictic centre can be maintained or updated during a discourse. the corpus contains 1,009 relations from five major relation types, which are a subset of the rst discourse treebank @xcite . in the rst-cc, relations are annotated with respect to @xcite ) seven continuity dimensions. the relations are furthermore annotated for additional features such as polarity (positive or negative relation) and context (intra-vs. inter-sentential relation). the paper is structured as follows: section 2 outlines previous work on continuity and discourse relations. in section 3, we describe the composition of the rst-cc, its general format, and the selected relations. section 4 elaborates on the selected continuity dimensions and their operationalisation into distinctive features. additional features of annotation are described in section 5. section 6 reports on the inter-annotator agreement study, and section 7 discusses recurrent annotation issues. we present first results in section 8, and conclude with an outlook on the next steps of our work. ## the rst continuity corpus the rst-cc comprises relations from the rst discourse treebank or rst-dt @xcite . the rst-dt contains 385 newspaper texts annotated for over 20,000 relations according to rhetorical structure theory or rst @xcite . in rst, relations link a more and a less central discourse unit (nucleus and satellite), or two equally central units (nuclei). linking is recursive, which models discourse as a tree structure. elementary discourse units (edus) in rst are typically clauses; there may be sub-clausal edus units, however (especially in the rst-dt). vated by previous classifications, which categorise, for example, causal and elaboration relations as continuous @xcite , contrastive relations as discontinuous @xcite , temporal relations as one or the other @xcite , and conditional relations as underspecified with respect to continuity @xcite . the relations are also classified in terms of the cognitive approach to coherence relations or ccr @xcite , using features such as polarity (positive or negative) foot_2 and basic operation (implicational or additive, i.e., causal or non-causal). for instance, elaboration relations are positive and additive, whereas contrastive relations are negative. in the rst-cc, the relation types are subdivided according to the rst-dt relation taxonomy @xcite ; e.g., the contrastive type includes the subtypes antithesis, conces-sion, and contrast. each relation was independently annotated by the two authors for the seven continuity dimensions. any differences were subsequently adjudicated before including the relation in the corpus. ## operationalising continuity dimensions to annotate the relations in the rst-cc according to @xcite seven continuity dimensions, we operationalised them into distinctive features foot_4 . ## additional features features that potentially influence the relationship between discourse relations and continuity are also annotated in the rst-cc. we include the ccr features polarity (see section 3) and order of segments. the latter applies to implicational (causal and conditional) relations only: the order is basic if the cause or antecedent segment precedes the result or consequent segment @xcite ; the reverse order indicates a non-basic relation. the relations are annotated for two more features: nuclearity (which specifies the segment pair as nucleus-satellite, nucleus-nucleus, or satellitenucleus, according to rst) and context (whether the relation occurs intra-or inter-sententially). the annotation scheme for the additional features is summarised in for illustration, we provide an example of the rst-cc annotation (for seven continuity dimensions and also for four additional features) in ## reliability of annotation to assess the quality of our annotation, we conducted an annotation experiment. for the seven continuity dimensions, we independently annotated a selection of 240 relations, which are not part of the rst-cc, but represent the five relation types of the corpus. agreement was substantial according to cohen's kappa @xcite for the four dimensions time, reference, perspective, and modality, as shown in to annotate the action dimension, we had to consult external encyclopaedic sources, since there were no script data available for the events described in the corpus data. however, our results show that for specialised domains like the economic topics featured in many articles of the rst-cc, external sources can greatly contribute to safeguarding inter-rater agreement. the scores reported for time and reference measure agreement on the binary distinction between continuous and discontinuous relations, as described in section 4. however, we also calculated scores for more fine-grained classifications. for reference, our annotation of the entire sevenfold classification of centering theory also yielded substantial agreement (κ = 0.62). the confusion matrices reveal that agreement is especially high for the preservation, the termination, and the lack of reference continuity (continue, zero, and null, respectively). we interpret this result as confirming the usefulness of centering theory for practical annotation initiatives. for time, we annotated a more fine-grained classification into non-temporal/synchronous, chronological, and anti-chronological constellations. agreement on this classification was only moderate (κ = 0.49). subsequent evaluation showed that the problematic distinction was the one between synchronous and chronological, in particular, for implicational relations. the choice of the values for temporal continuity varied over whether the consequent (or result) starts simultaneously with the antecedent (cause) or whether the latter follows the former. the issue is illustrated in ( 21 ), for which one annotator assumed that the junk market getting its biggest jolt (cause) is synchronous with it going into a tailspin (consequence), whereas the other one understood a chronological order in that the tailspin began after the jolt. subsequent discussion of these decisions revealed that the forced choice between the two possible temporal constellations introduced considerable arbitrariness, which was reflected in low agreement. consequently, one should avoid forcing a choice in these cases by subsuming the two constellations in the underspecified statement that the consequence does not precede the antecedent. we conclude that such examples pose a severe challenge for approaches to temporal continuity that, unlike ours, regard chronological (as opposed to synchronous) order as non-continuous. for perspective, the high agreement was supported by the fact that newspaper text indicates the sources of direct or indirect quotes very clearly. the disagreements mainly involved distinguishing reported facts from any kind of comment or conclusion drawn from them. for other text types, we envisage that the identification of perspectives must take into account additional linguistic evidence, e.g., in the case of free indirect discourse @xcite . ## recurrent annotation issues this section presents recurring issues for our annotation which make choosing the correct label for a specific continuity dimension challenging. ## first results we provide the distribution of continuous relations (proportions in percentages) for five relation types with respect to the seven continuity dimensions in we found that some continuity dimensions show uniformity across relation types. relations of all types are found to be overwhelmingly continuous (> 98%) for the dimensions space and speech act, and almost never continuous (< 2%) for action 7 . we believe that this is due to our data: in particular, the non-narrative character of our data is responsible for the low degree of action continuity and 7 however, even these dimensions exhibit 100% continuity or discontinuity for a specific relation type only rarely: elaboration is 100% continuous for space, contrastive, 0.00% for action, and 100% for speech act. for the high degree of space continuity. in addition, there are very few questions and imperatives in our newspaper data, which explains the overall continuity for speech act. due to these limitations of our corpus, we believe that the uniformity we found for the space, speech act, and action dimensions does not suggest that these dimensions are less important for continuity in discourse relations; instead, they might become distinctive if material from other registers is investigated. for the dimensions time, reference, perspective, and modality, however, there is considerable difference between the relation types, as summarised in continuity scores for reference are consistently lower for two reasons: there are many small discourse segments in the rst-dt, which reduces the chance of finding a shared referent across the segments. this is illustrated by ( 27 ), where the target relation, consequence-n, holds between segments a (the single word 'lost') and b. (27) [mr. lagnado said] [that] [although retailers probably won't ever recover sales] [lost] a [because of the california quake and hurricane hugo,] b [they could see some benefits later on.] since neither of the two segments has a background-looking centre (cb), referential continuity of the relation is calculated as null, which amounts to discontinuity. moreover, we did not consider eventive and propositional referents in the analysis. however, as long as we compare only reference scores across the relation types (or subtypes), this will not affect our results. the correlations between relation types and continuity along a specific dimension are significant at @xmath0.001 (@xmath1.05 for reference) for all dimensions except space and speech act. ## conclusions and outlook we presented the rst continuity corpus (rst-cc), which comprises five major types of discourse relations annotated for a wide array of continuity dimensions and additional features. we envisage two applications of the corpus. first, the rst-cc will contribute to a more precise characterisation of discourse relations, providing a systematic, detailed, and reliable resource for examining the relationship between continuity (dimensions) and discourse relations. in addition, the corpus can also be used to test hypotheses about correlations between continuity dimensions and discourse relations. for example, contrastive relations often present information about different (though comparable) items or information from different sources, and one can test whether this would lead to low scores for reference and perspective continuity. second, the corpus, in conjunction with parallel resources like the rst signalling corpus @xcite , will contribute to the study of discourse signalling, e.g., to explore the continuity hypothesis @xcite , which entails that discontinuous discourse relations are harder to process, and hence, their processing should be facilitated by more explicit signalling. furthermore, it is an important research question whether continuity in discourse relations patterns uniformly or differently across genres or languages. for further work in this field, the development of the rst-cc could be a model for similar resources for different genres and different languages (other than news texts in english, as in the rst-cc). finally, we believe that our decompositional approach towards continuity would support further in-depth analyses of discourse relations. the varying effect of different continuity dimensions on discourse relations, for instance, would help resolv-ing incongruities found in the study of discourse processing (why certain discourse relations are processed quicker and remembered better than others). for a broader empirical basis for such investigations, we will extend the rst-cc, adding more instances of the relation types covered so far, but also including additional relation types like