The ARRAU 3.0 Corpus (2024)

Abstract: The ARRAU corpus is an anaphorically annotated corpus designed to cover a wide variety of aspects of anaphoric reference in a variety of genres, including both written text and spoken language. The objective of this annotation project is to push forward the state of the art in anaphoric annotation, by overcoming the limitations of current annotation practice and the scope of current models of anaphoric interpretation, which in turn may reveal other issues. The resulting corpus is still therefore very much a work in progress almost twenty years after the project started. In this paper, we discuss the issues identified with the coding scheme used for the previous release, ARRAU 2, and through the use of this corpus for three shared tasks; the proposed solutions to these issues; and the resulting corpus, ARRAU 3.

## introduction although the scope and ambition of anaphoric annotation projects has enormously increased in the last twenty years @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite a number of open questions about anaphoric annotation remain, and many if not most of the existing corpora have limitations either in size or coverage. the arrau annotation @xcite @xcite is a long-term project to expand the range of anaphoric annotation by creating an anaphorically annotated corpus covering a wide variety of aspects of anaphoric reference @xcite . the annotation project started in 2004 as the result of a series of studies of the reliability of 'difficult' as-pects of anaphoric annotation @xcite @xcite and the first release was primarily focused on anaphoric reference in dialogue @xcite . the scope of the annotation then broadened both in terms of linguistic aspects that were annotated and in terms of genres, resulting in a second release in 2013 @xcite . this second release was then used as the core dataset for the 2018 crac shared task @xcite , the first shared task for anaphora resolution covering also identification of non-referring expressions, bridging reference and discourse deixis; and as additional material for the 2021 and 2022 codi-crac shared tasks on anaphora resolution in dialogues @xcite . these shared tasks highlighted the need to revise the annotation guidelines for a range of phenomena including discourse deixis and genericity and reference in dialogues. they also revealed a number of issues with tokenization and markup. we therefore started an extensive reannotation and cleaning up, resulting in a third, substantially revised release of the corpus. in this paper we discuss the issues identified with the previous annotation, the revised annotation scheme and guidelines, the cleaning up procedure, and the new corpus resulting from this effort. ## anaphoric annotation we review in this section the aspects of anaphoric interpretation captured in the arrau annotation. identity anaphora most modern anaphoric annotation projects cover identity anaphora as in (1). however, many other types of identity anaphora exist, as well as other types of anaphoric relations, discussed below. split-antecedent anaphora in most corpora, plural reference is only marked when the antecedent is mentioned by a single noun phrase. but in splitantecedent anaphors @xcite such as (2), plural pronoun they refers to a set composed of two entities introduced by separate noun phrases. event anaphora is a subtype of the more complex phenomenon of discourse deixis @xcite which also includes references like this in (4), which refers to the fact that the rabbit was able to talk. not many corpora attempt to cover the entire range of discourse deixis. bridging references and other non-identity anaphora possibly the most studied type of nonidentity anaphora is bridging reference or associative anaphora @xcite @xcite as in (5), where bridging reference the roof refers to an object which is related to / associated with, but not identical to, the hall. other types of non-identity anaphora also exist, besides bridging references. examples include other anaphora like the other in (6), as well as identity of sense anaphora such as a blue one in (7) @xcite . the interplay between anaphora and other semantic properties of nominals often, whether two mentions corefer depends on how they get semantically interpreted in other respects. in (8), for instance, whether the mention bananas in 40.2 is interpreted as coreferring with mention bananas in 37.8 depends on whether these bare plurals are taken to be references to the generic kind bananas @xcite . if those mentions are interpreted as non-generic, they would not corefer. some anaphoric corpora therefore include an annotation of noun phrases' genericity @xcite . anaphoric reference in dialogue anaphora resolution in dialogue requires systems to handle grammatically incorrect language suffering from disfluencies and mentions jointly created across utterances @xcite or whose function is to establish common ground rather than refer @xcite . dialogue contains more deictic reference, vaguer anaphoric and discourse deictic reference, or speaker grounding of pronouns. these complexities are normally absent from news or wikipedia articles, which form the bulk of current datasets for coreference resolution @xcite . there has been some research on coreference in dialogue in english @xcite @xcite , but very limited in scope (primarily pronominal interpretation), due to the lack of suitable corpora, although the situation is better for other languages @xcite . ## arrau 1 and 2 the arrau corpus foot_0 @xcite was designed to cover a variety of genres. initially, the corpus was meant to focus on anaphoric reference in dialogue and spoken language @xcite . its trains sub-corpus includes all the task-oriented dialogues in the trains-93 corpus foot_1 @xcite already used in byron's work on pronominal reference in dialogue @xcite as well as the pilot dialogues in the so-called trains-91 corpus. the pear sub-corpus consists of the complete collection of spoken narratives in the pear stories that provided some of the early evidence on salience and anaphoric reference @xcite . 3 subsequently, the corpus was extended to cover a substantial amount of written text, including news text in a sub-corpus called rst, consisting of the entire subset of the penn treebank @xcite that was annotated in the rst treebank @xcite . 4 the gnome sub-corpus covers documents from the medical and art history genres covered by the gnome corpus @xcite . ## arrau 3: summary of the revisions the crac 2018 shared task revealed a number of issues with the arrau 2 annotation -first of all with the annotation of bridging references and discourse deixis-that prompted a first round of revisions to the annotation scheme and the annotation guidelines. more issues about the annotation of anaphoric reference in dialogue were revealed when the data were used for the codi-crac 2021 shared task, resulting in a second round of revisions. during the codi-crac shared task we also discovered issues with tokenization and with the way the rst portion had been converted. as a result, we started revising the corpus by: (i) revising annotation schemee and guidelines (ii) fixing the issues with tokenization and with conversion. in the following two sections, we discuss each of these revisions in detail.