Developing a multilayer semantic annotation scheme based on ISO standards for the visualization of a newswire corpus (2021)

Abstract: In this paper, we describe the process of developing a multilayer semantic annotation scheme designed for extracting information from a European Portuguese corpus of news articles, at three levels, temporal, referential and semantic role labelling. The novelty of this scheme is the harmonization of parts 1, 4 and 9 of the ISO 24617 Language resource management - Semantic annotation framework. This annotation framework includes a set of entity structures (participants, events, times) and a set of links (temporal, aspectual, subordination, objectal and semantic roles) with several tags and attribute values that ensure adequate semantic and visual representations of news stories.

## introduction the development of an annotation framework can be an overwhelming task, even more when its purpose is to account for different linguistic phenomena. however, as challenging as it may be, designing an annotation scheme is an indispensable step to generate language resources that can be the starting point of fundamental corpus-based linguistic research. when deciding on an annotation framework, one has to take into consideration several factors @xcite , such as main objectives of the annotation, the linguistic phenomena under analysis, the corpus genre, and the nature of the annotation, and weigh in the advantages and 1 https://text2story.inesctec.pt/ disadvantages of adapting/ adopting an existing model, or of creating one. ideally, the model is custom designed to deal with all the specificities of a particular project, but also broad enough so that it can be applied to other datasets. in fact, with the growth of the semantic web and linguistic linked data @xcite , interoperability is key to read and to interpret linguistic resources @xcite . with all the above-mentioned provisos in mind, we developed a multilayer semantic annotation scheme by combining three standards from the language resource management-semantic annotation framework: part 1-time and events (iso-24617-1), part 4-semantic roles (iso-24617-4) and part 9-referential annotation framework @xcite . in addition to promoting interoperability, our model has proven to be able to markup manually the relevant features of the genre news to generate visual representations of their narratives. moreover, our proposal operationalizes the integration of three different standards in the same framework, which is, to the best of our knowledge, a novelty. this multilayer semantic annotation scheme was designed to annotate a european portuguese corpus of news articles in three different, but complementary, levels, temporal, referential and thematic, within the text2story project 1 , which aims to extract narratives from news, represent them in intermediate data structures, and make these available to subsequent media production processes, i.e., visualizations such as message sequence charts (msc) and knowledge graphs (kg). in this paper, we document the decision-making process about which annotation format to adopt, what adjustments to make, and how to harmonize the three layers into an integrated and wide-ranging model. ## background and motivation news may frequently assume the format of a story that reports on current events involving one or more entities in given time and place. in addition to the main event, however, news stories typically present contextual content that allows connecting it to others, explaining the circumstances and consequences of its occurrence. it may also include other complementary information that frames, comments, clarifies, or evaluates the reported events @xcite @xcite . a complete story usually answers six questions: what, who, where, when, why, and how, that is, 5w1h (a.o. bonet-jover et al., 2021), following a top-down organization, corresponding to an inverted pyramid discourse structure (cf. @xcite , in which information flows in decreasing order of importance. a news organization structure usually features a title, a lead, and the body. in many cases, the lead or introductory paragraph condenses the answers to the above six questions and is followed by complementary information (a.o. @xcite . sometimes, the answer to some of the questions is distributed throughout the text @xcite . because of this organization, events frequently follow a non-chronological order, presenting a complex time structure regarding other kinds of narratives @xcite . besides, the narrative may return to previous data, as well as adding information (a.o. @xcite @xcite . establishing the temporal sequencing of events, their participants, and interrelations is crucial to understand the news story, and ultimately to extract the narratives to be represented graphically by means of msc @xcite or kg @xcite , which is our project's main objective. these visualizations by portraying the narratives more schematically can be of great interest to news agencies, for example. the more overarching and rigorous the annotation the more informative is the visualization, and, in the case of news articles, this requires featuring participants, events and times, as well the relationships between them. for these reasons, the annotation scheme that we designed encompasses three intertwined semantic layers: temporal, referential and thematic. since our aim was to adopt a coherent and interoperable annotation scheme with these three layers, and because none of the existing proposals satisfied these requisites, we designed an annotation scheme which compatibilizes three iso. ## related work over the last years, there has been a proliferation of multilayer corpora, that is, corpora that "contain mutually independent forms of information, which cannot be derived from one another reliably" @xcite . these layers can be defined in an independent way and they "are explicitly analyzed using multiple, independent annotation schemes" @xcite , or resorting to one unique scheme that integrates all the layers. in fact, an indepth analysis of the relevant literature reveals that there are many different types of multilayer annotation schemes. in the remainder of this section, we will only present a brief overview of some of those proposals. one of the most well accomplished and farreaching multilayer annotation schemes is the one developed within the groningen meaning bank (gmb) @xcite . besides morphological and syntactic annotation, it comprises different semantic annotation levels, such as named entity recognition, temporal features, and thematic roles. the adopted semantic formalism is an extension of discourse representation theory @xcite , which renders a semantic representation (discourse representation structures) that unifies the various layers. another important feature of this scheme is that it was designed to analyze linguistic phenomena in texts, instead of only sentences, and it has been used quite successfully in 10,000 texts from different genres, namely news and fables. its implementation requires a human-aided machine annotation insofar as it employs nlp software such as an automatic tagger for named entity recognition, verbnet @xcite for semantic role labelling, a semantic analyzer for coreference, and then a module boxer @xcite @xcite , responsible for the overall semantic analysis, but also relies on the input of experts and general public. although, in terms of semantic annotation, it is one of the most complete, this scheme lacks information about more referential relations. moreover, since the temporal annotation is based on drt-language, it does not integrate tags about lexical and contextual meaning with bearing on temporal interpretation, namely a more diversified class of events, and other link types between events. other multilayer annotation schemes have been developed for manually annotated sub-corpus (masc) @xcite , georgetown university multilayer corpus (gum) @xcite , ontonotes @xcite , for amalgum @xcite , or sensem (fernández and vázquez, 2014), just to name a few, but none of those provide a comprehensive and harmonized semantic framework suitable to handle the linguistic phenomena that we need to address. for european portuguese (ep), one can point out the scheme used in cintil deepbank @xcite , which is a corpus of portuguese news and novels that is annotated with several grammatical information (morphological, syntactic, and semantic) for each sentence. currently, there are 32497 sentences, mainly from news, which were semi-automatically annotated with treebank, dependencybank, propbank, and logicalformbank (with formal representations of the sentences meanings using minimal recursion semantics). however, the cintil deepbank's scheme does not include a level for referential annotation, nor for temporal annotation. the fact that only the sentences that the grammar can parse are included in the corpus is a downside. additionally, though each level of annotation can be accessed separately, a unifying formalism that combines all the layers is missing. regarding schemes aimed exclusively at semantic annotation, some are intended to handle a specific phenomenon, resort to non-standardized markup language, and are not widely known (cf. for an overview @xcite . moreover, the majority deals with lexical problems, such as word disambiguation, and less with compositional semantics. the scarcity of proposals within this branch of semantics can be explained by the complexity underlying the process of annotating semantically data at a sentential and textual level. this task requires not only a great amount of time, but also a wide variety and substantial number of resources. nonetheless, semantic schemes to represent the meaning of texts are of utmost relevance to the development of different applications. ## an annotated example in our model, the annotation procedure consists of three stages. example (1) will serve to illustrate the three stages. in the first stage, the annotator marks the entity structures of events and times, and, then, the temporal, aspectual and subordination links are established. events @xmath0, the participants are identified, and they are related to each other by objectal links. participants @xmath1 in the third stage, the annotator connects participants to events by semantic role links. ## conclusion in this paper, we present an annotation framework for news articles in ep that aims to provide the input for visualization processes. first, we determined what type of information was necessary to account for events and participants in the narratives, and decided that three annotation layers -temporal, referential and thematic -were required. the next step was to decide which tags and links should be used in each layer to fulfill the annotation purposes. since interoperability is crucial when we talk about semantic resources, three standards iso 24617-1/4/9 were utilized to create a multilayer semantic annotation scheme. notwithstanding the fact that these standards are, in fact, themselves three parts of the same standard, when combined, some inconsistencies arise. so, we had to harmonize the three layers, to attain a cohesive annotation framework. additionally, we sought to balance the amount of information needed to capture the news stories and the load of the annotation process. although this model was built to capture the structure of stories in news in ep, its scope is not limited to news nor to ep, as it can be extended to other narrative texts and other languages with some adaptations to deal with genre and language specificities. moreover, the integration of three different layers in a single annotation framework enables formal semantic representation with drs, which acts as an intermediate language to generate visualizations in the form of knowledge graphs, for instance. in the future, we intend to endow our annotation scheme with more granularity. to this end, iso standard for spatial information (iso 24617-7) will be added to our framework. for now, spatial annotation has relied on the tags, attributes and links available in the referential and thematic layers. likewise, a more detailed information regarding quantification of participants and of events is a component to be improved in the future. at this moment, this kind of information has a very simplified representation solely in the referential layer, which does not fully represent the different possibilities of quantification over entities.