Reference and discourse structure annotation of elicited chat continuations in G erman (2024)

Abstract: We present the construction of a German chat corpus in an experimental setting. Our primary objective is to advance the methodology of discourse continuation for dialogue. The corpus features a fine-grained, multi-layer annotation of referential expressions and coreferential chains. Additionally, we have developed a comprehensive annotation scheme for coherence relations to describe discourse structure.

## introduction elicited discourse has been used as a source of data for the study of language, communication and the human mind in a variety of settings. while the data are inevitably less 'natural' than in naturally occurring discourse, elicitation makes it possible to observe discourse phenomena under more or less controlled conditions and to test specific hypotheses that might be difficult to test in the wild. the degree of control of the context in which the discourse is produced ranges vastly from giving just a general direction of the communication (e.g. the switchboard corpus, @xcite , to asking the participants to fill in gaps in a preconceived template (e.g. @xcite . discourse continuation experiments clearly belong to the higher end of this spectrum: the participants are given the beginning of a story (one or a couple of sentences), and are asked to continue the story by writing one or more sentences. over the years, this method has established itself in the study of the interpretation and the production of referring expressions @xcite , as well as the interaction between reference and coherence relations-meaningful links between sentences and clauses that represent the function of each sentence or clause in the text and ultimately make us perceive the sequence as a coherent whole @xcite . however, the method has been used almost exclusively to elicit monologue, and experimental dialogue continuation studies are few (e.g. @xcite . typically, dialogue is spoken, whereas discourse continuation is easier to implement as a written task. although spoken discourse continuation elicitation studies also exist (e.g. @xcite , combining the spoken mode with the interactive setting of dialogue makes the task much more challenging. (for instance, in a situation with multiple speakers, how do we reliably make sure that the participant continues in the role of the right speaker?) one of the main goals behind the creation of the present corpus was to extend and further develop the discourse continuation methodology in application to dialogue. however, we wanted to avoid the complexities of the elicitation and the analysis of spoken data, and at the same time to make the task as natural as possible for the participants. thus, we chose chat as the dialogue form for our elicitation experiment. since the advent of smartphones, written dialogue has become an everyday activity for a vast majority of adult population @xcite , so we decided to rely on people's familiarity with common instant messaging applications such as whatsapp, telegram, and skype, and frame the task as chat continuation. furthermore, there is a constant need for the development of high-quality annotated corpora that encompass a diverse range of languages, language uses, and genres in the nlp world. traditionally, nlp resources have relied heavily on formal written sources such as wikipedia and newspaper articles, which, while valuable, represent only a portion of human communication inventory. this issue has led to a growing awareness of the need to include more varied forms of language, especially those that mirror everyday communication, like spoken conversations and chat messages. chat messages offer a blend of the immediacy and informality of spoken language with the structured format of written texts. however, they also bring unique challenges, such as handling the excessive use of abbreviations @xcite , emojis @xcite , non-standard grammar @xcite , and slang @xcite that are common in chat communication. as with the spoken data, another significant challenge is ethical considerations, particularly in privacy and data protection. chat messages often feature personal communications, necessitating strict anonymization rules and adherence to data protection laws such as general data protection regulation (gdpr). the present corpus circumvents these problems by eliciting chat conversations in an experimental setting, which allows us to preserve the spontaneous and informal nature of real chats while avoiding legal and ethical concerns. in section 2, we present an overview of available corpora for the study of reference, coherence relations, and previous efforts in creating chat corpora. section 3 details the data collection procedure. section 4 sets out the motivation for our annotations and the annotation plan we pursued. in section 5, we show the application of the annotation by presenting case studies. ## related work reference-annotated corpora differ from one another in the genre and modality of the texts they include. some are specifically designed for studying reference, while others serve more general purposes. another difference is in the nature of their source materials-whether these corpora are based on pre-existing texts or are compiled from data gathered in systematically designed experiments @xcite . ontonotes @xcite and gum @xcite are examples of general-purpose corpora with layered annotations, including syntax, part of speech, and coreference. ontonotes features a range of genres, from news to phone conversations, and provides extensive annotations of coreferential chains. however, it does not specifically annotate the type (e.g., pronoun, proper name, or definite np) of each referring expression. gum offers broader annotation scope, including detailed reference and rhetorical structure annotations in discourse. on the other hand, grec-2.0 and grec people @xcite are specialized for analyzing referring expressions within context, derived from wikipedia article introductions. they provide extensive annotations on the form and grammatical role of referents, but grec-2.0 is limited to annotations related to the main subject, and grec-people exclusively annotates human referents. in addition to the above corpora, which are constructed from existing resources, there exists also a range of corpora developed in experimental settings for the study of reference. these corpora, including scare @xcite , coconut @xcite , tuna @xcite , g-tuna @xcite , give-2 @xcite , and pentoref @xcite , are derived from elicited language in controlled settings like virtual reality games and computermediated dialogues. these corpora, involving tasks like instruction-giving or furniture-buying, predominantly feature short exchanges about inanimate objects. consequently, their annotations focus almost exclusively on inanimate entities, disregarding annotations for animate referents. most corpora annotated with coherence relations, such as the penn discourse tree bank @xcite and the german potsdam commentary corpus @xcite consist of written texts, primarily newspaper articles. stac is an example of a dialogue corpus with extensive coherence annotations, containing chats between the players in an online settlers game @xcite , whose approach to annotation we selectively adopt in the present project. the corpus showcased in our study combines informal (non-task-oriented) written dialogue with comprehensive annotations of both animate and inanimate referents, and coherence relations, essential for understanding natural communication. ## annotation most of the annotations were conducted by two of the authors of the paper. our annotation scheme was reexamined on a continuous basis, leading in some instances to changes of earlier annotation phases. the annotations were conducted with the web-based annotation software inception @xcite , see appendix a for an example. we annotated referring expressions (re) and coreference relations between them (section 4.1), identified elementary discourse segments and their types (section 4.2), as well as the coherence relations between them (section 4.3). ## studies in the following, we show examples of analyses that can be performed using our corpus. section 5.1 presents a number of quantitative characteristics of the corpus, whereas section 5.2 illustrates some observations based on the qualitative analysis of the elicited chats. ## discussion and conclusion to summarize, we have created a corpus of german chats elicited in an experimental setting, which can be used to study the choice of referring expressions (res) and coherence relations in naturalistic language use. we find the creation of this corpus and the extensive annotations we have conducted valuable for the following reasons: (1) the data has an informal dialogic nature in a simulated interactive setting, that represents an adaptation of the established discourse continuation task to dialogue; (2) since it is an experiment, there is data available from multiple participants for the same scenarios, enabling the study of individual variation, as well as quantitative generalizations over comparable structures; (3) in addition to the annotation of coreference, the corpus presents an extensive annotation of referring expression forms; (4) the data contains an annotation of coherence relations to give a better picture of referring in naturalistic settings; (5) the data has been collected for the german language, however, the experimental setting is easily expansible to other languages, which makes cross-linguistic comparisons possible. the raw and annotated data, along with the annotation guidelines, are publicly available and can be accessed on our github repository: https: //github.com/yli671/germanchatcorpus