Interlingual Annotation of Multilingual Text Corpora and FrameNet

This article raises an issue of common interest to those interested in Interlinguas and interlingual MT and those interested in developing a multilingual FrameNet. Specifically, it addresses the problem of teasing apart the difference between meaning and interpretation, between semantics and pragmatics and between semantic representation and the representation of conveyed information. No translation (nor paraphrase) conveys the exactly same information as the original utterance. Rather, additional information may be conveyed or information may be lost, and information originally expressed explicitly may be conveyed implicitly and vice versa. The semantic representation of an utterance (the result of integrating the semantic representations of the it subcomponents) does not capture what people intuitively feel is the meaning of an utterance but rather various pragmatic factors must be taken into account as well including the time and place of utterance and the speaker’s motivation for uttering. The focus of the discussion is on describing IAMTC, a multi-site NSF-supported project to annotate six sizable bilingual parallel corpora for interlingual content. After setting out the basic issues, we present the background and objectives of the IAMTC annotation effort, the dataset being annotated, the interlingual representation language used, the annotator’s interface and annotation process itself, along with the evaluation methodology and results of an initial evaluation. Finally, we conclude by summarizing the current state of the project and presenting a number of issues yet to be resolved. as372@cl.cam. ac.uk 1 Translation, Meaning and Interpretation The importance of linguistically-annotated parallel corpora and multilingual annotation tools is now widely recognized (Véronis 2000), yet there are currently few cases of annotated parallel corpora, and those which exist tend to be bilingual rather than multilingual (Garside, et al. 1997). Moreover, much of the previous work on the linguistic annotation of corpora has focused on the annotation of sentences with syntactic information only, e.g., part-of-speech tags (Brown Corpus (Francis & Kucera 1982)) and syntactic trees (Penn Treebank (Marcus, et al. 1994)). Even where the focus is on semantic representation as in the case of PropBank (Kingsbury & Palmer 2002), NomBank (Meyers, et al. 2004) or the FrameNet example corpus (Baker, et al. 1998), the corpus has generally been monolingual. Two exceptions to this general state of affairs are the multilingual FrameNet (Boas 2005) and the IAMTC project (Interlingual Annotation of Multilingual Text Corpora). In the case of multilingual FrameNet, a large corpus of sentences exemplifying and annotated for semantic frames and the relevant frame elements are being translated and annotated in a number of other languages, in principle creating a large multilingual parallel annotated corpus. In the case of the IAMTC project, six different but comparable corpora, each consisting of a set of source language news articles along with two or three independently produced manual English translations, are being annotated for interlingual (IL) content. Viewing semantic frame representations as interlingual representations, it would appear that the two projects are essentially the same, annotating parallel corpora for interlingual content. This, however, is not precisely the case. Interlingual approaches to machine translation are based on the assumption that there is a level of utterance representation at which all the relevant aspects of information needed for generating an equivalent utterance (i.e., a translation in a second language or a paraphrase in the same language) can be captured. Similarly, multilingual FrameNet developers assume that there is some level of representation, the semantic frame, at which all aspects of information relevant to the description of the lexical content of a set of related predicates can be captured both within and across languages. Thus, both efforts attempt to represent “aspects of information. ” Just as providing atravesar el río nadando as a translation of to swim across the river depends on both expressions sharing a common interlingual representation, which can be broadly represented as: MOVE (MODE SWIM) (ULTERIOR-SURFACE-CONTACT RIVER), so too providing to cross the river swimming as a paraphrase of to swim across the river is based on both having the same frame representation, loosely: MOVE (MODE SWIM) (ULTERIOR-SURFACE-CONTACT RIVER). To the degree that IL representations must represent semantic content, then, both efforts seek an abstract representation of event-types commonly referred to by predicates – or a lexical semantic description for related verbs (e.g., verbs of commercial transaction). They differ only in that, for translation, the criteria for motivating a given representation are based on cross language 1 IAMTC has been supported by NSF ITR grant IIS-0325887 correspondences whereas, for paraphrasing, the criteria for selecting a given representation are based on maintaining semantic equivalence within the language. But interlingual representations and semantic representations are not concerned with exactly the same “aspects of information. ” IL captures interpretations rather than simply denotational content. So, for instance, the IAMTC annotator is faced with deciding whether earthquake predictions and predicted earthquakes should be provided with the same representation and, if so, what representation, since they appear as alternative translations of anuncios sismicos (seismic warnings). Similar decisions must be made in regard to assassin and murderer as variant translations of asesino in reference to a policeman on trial for killing a union organizer, to third floor or fourth floor as legitimate alternative translations of tercer piso (lit. third flood) in a European Spanish text translated for US English speaking audience (because of different conventions for naming the levels of a building), or to started its business and opened its doors to customers as alternative translations empezaron el negocio. This means that it must capture the intended meaning of non-literal language as well as literal meaning. In addition, it means that IL must capture pragmatic information concerning the organization of the speech act (topic/focus, and so on). In regard specifically to the two annotation efforts, the original FrameNet dataset is in fact monolingual. It consists of isolated English sentences selected because they exemplify some aspect of an items frame structure. The resultant multilingual corpus consists of translations of that original dataset. For IAMTC, on the other hand, the dataset consists of two or three independently created translations in the same language (English) along side the original source language text. The texts are news articles consisting of multiple consecutive sentences and generally 300 words long or so. The news articles are randomly selected and may not exemplify anything in particular. Annotation proceeds by comparing translations, categorizing any differences (as errors, paraphrases or meaningful variations, reflecting information loss or gain) and especially in the case of meaningful variations, identifying the inferences and knowledge needed to produce that variant. The representations themselves differ as well. Originally, frame representations are motivated by morphosyntactic criteria related to non-meaning changing paraphrases. Less clear are the criteria that apply in deciding whether expressions bear some other potential lexical relation when they are associated with the same “metaframe” (e.g., conversives buy and sell to the “commercial transaction” metaframe). The IAMTC IL is the result of successive abstractions away from surface form: • syntactic dependency structures (normalized for cross-linguistic consistency between Arabic, English, French, Hindi, Japanese, Korean, Spanish and across translations), • semantically enriched with ontological predicates and semantic relations (normalized as above), • and “abstracted” merged meaning representations. IL then captures the “intended” semantic structure along with the inferences (and knowledge) used to arrive at that representation. It is expected that a broader range of “paraphrases” will be represented similarly because analysis is at the clause, sentence and, in some cases, paragraph levels as opposed to the lexical level. In what follows, we will focus on presenting a more detailed description of the IAMTC project without dedicating much discussion to the similarities and differences between our project and the multilingual FrameNet effort. We assume rather that the reader will be able to compare the two and determine how the efforts might inform one another. In Section 2, then, we introduce the objectives of the IAMTC project and provide some background. In Section 3, we describe the corpus and, in Section 4, we present the IL representation scheme and supporting resources. In Section 5, we describe the annotation methodology and tools. In Section 6, we present an evaluation methodology and present the results of an initial evaluation. Finally, in Section 7, we conclude with a discussion of the achievements thus far and point out a number of issues that have arisen or have yet to be addressed. 2 The Interlingual Annotation of Multilingual Text Corpora (IAMTC) Project With the recent shift toward deeper, corpus-based acquisition of language-independent representations (Hovy, et al. 2003), the next step is to provide a significant foundation for more sophisticated language-processing techniques. The IAMTC project focuses on that next step: the creation of a system of text meaning (or interlingual) representation and the development of a number of sizeable semantically-annotated parallel corpora, for use in applications such as machine translation and a range of other natural language applications