E vent N et- ITA : I talian Frame Parsing for Events (2024)

Abstract: This paper introduces EventNet-ITA, a large, multi-domain corpus annotated full-text with event frames for Italian. Moreover, we present and thoroughly evaluate an efficient multi-label sequence labeling approach for Frame Parsing. Covering a wide range of individual, social and historical phenomena, with more than 53,000 annotated sentences and over 200 modeled frames, EventNet-ITA constitutes the first systematic attempt to provide the Italian language with a publicly available resource for Frame Parsing of events, useful for a broad spectrum of research and application tasks. Our approach achieves a promising 0.9 strict F1-score for frame classification and 0.72 for frame element classification, on top of minimizing computational requirements. The annotated corpus and the frame parsing model are released under open license.

## introduction frame parsing is a powerful tool for real-world applications in that it enables deep grasp of the meaning of a textual statement and automatic extraction of complex semantic descriptions of situations, including events, and their relations with the entities involved. to this effect, frame parsing can effectively be used for event extraction, as the two tasks share the common goal of recognizing and classifying argument structures of a target predicate. however, training supervised models for frame parsing is a data-intensive task, which is why comprehensive linguistic resources are available in few languages, thereby limiting further research and application in downstream tasks. furthermore, most existing corpora are created by targeting the annotation of one single frame (or event) class per sentence. while lexicographically motivated, this procedure makes the training of automatic models and their application to real-world scenarios more complicated and burdensome. the contribution described in this paper is twofold: first, we present eventnet-ita (evn-ita) a large-scale, multi-domain corpus annotated fulltext (see section 4) with over 200 semantic frames of events @xcite and 3,600 specific frame elements in italian, also discussing the motivation behind its creation, the annotation guidelines and the covered domains; secondly, we introduce an efficient multi-label sequential approach for eventive frame parsing and evaluate it on the dataset. this work aims at providing the community with a solid, manually-curated corpus and a ready-to-use model for frame-based event extraction for italian, thus filling an existing data gap. in fact, recent works in application fields like computational social science @xcite or historical nlp @xcite showed how semantic frames can be used as a powerful textual analysis tool to investigate a wide range of societal and historical phenomena. the annotated dataset, along with its full documentation, is released to the community under open license (see section 7). the envisioned application purpose of evn-ita is that of enabling accurate mining of events from large collections of documents, with focus on individual, social and, in a broad sense, historical phenomena. the paper is structured as follows: section 2 discusses existing work in frame parsing and event extraction, with a subsection focused on italian; section 3 introduces our annotated corpus and describes the motivations and design decisions that guided its creation, while section 4 focuses on the annotation procedure. in section 5 we discuss the methodology for frame parsing, a transformerbased multi-label sequence labeling approach. in section 6 we evaluate our methodology and discuss the results. section 7 provides the reader with pointers for the dataset and model release, while section 8 concludes the paper and highlights future directions of our work. ## related work the development of systems able to recognize and classify event mentions and their argument structure in text has been a long-term effort in computational linguistics and a variety of methods has been employed for the task of event extraction @xcite @xcite @xcite @xcite . event extraction is the task of recognizing and classifying event mentions and entities involved in the event from a textual statement and it has seen applications in a wide range of fields, like social media analysis @xcite , biomedical nlp @xcite @xcite , history and humanities @xcite @xcite @xcite , as well as literary text mining @xcite . although benchmark datasets exist, like automatic content extraction (ace) @xcite for event extraction or tac-kbp @xcite for multiple event-related tasks, they exhibit limitations in terms of size and domain coverage. also, while they are well suited for evaluation campaigns, they have not been designed for use in real-world application tasks. moreover, most of these corpora only exist for english, with few extensions for other languages @xcite . ## annotation the textual corpus, generated as discussed in section 3.1, has been manually annotated by labeling event triggers with their frame class and predicate arguments with the corresponding frame element. annotation was performed at the sentence level and was conducted frame-driven, by first selecting significant event frames for the domain and subsequently identifying the most relevant lexical units for each frame. given a sentence, any lexical units in our schema and all related frame elements are annotated, producing as many layers as there are event mentions (full-text annotation). figure 2 shows an example of full-text annotation from evn-ita. ## methodology a traditional approach for frame parsing, as well as for event extraction, is to break down the problem into sub-tasks @xcite , usually separating the steps of trigger identification, frame classification and argument extraction. however, a major downside of this approach, besides being more complex, is that it implies error propagation from higher-level sub-tasks downwards. instead, we propose to learn all the tasks in one single step, allowing the model to simultaneously exploit tag relations on the time (sequence) axis and on the token axis. thus, in this work frame parsing is approached end-to-end and is treated as a multilabel sequence tagging problem. the strength of this design option lies in its simplicity as it requires minimal pre-processing and does not imply the use of additional knowledge, as well as in its efficiency, as it minimizes computational requirements (see section 5.2). ## evaluation in this section, we discuss the quantitative (section 6.1) and qualitative (section 6.2) performance of the multi-label sequence labeling approach on the evn-ita dataset. ## dataset and model release the evn-ita annotated dataset, along with its documentation, is being released upon request 12 , under cc-by-sa 4.0 license foot_10 . the model of the frame parser, described in section 5.2, is available on huggingface foot_11 . ## conclusion and future works in this paper we presented evn-ita, a large corpus annotated with event frames in italian, accompained by an efficient multi-label sequential model for frame parsing, trained and evaluated on the corpus. future work includes extrinsic tests of the resource on new data from different textual genres and the reinforcement of the schema, in view of providing a wider domain coverage and increased adaptability of the model. moreover, is our plan to employ evn-ita as a benchmark to investigate the performance of different methodologies and learning models for frame parsing, as well as to explore strategies for multilingual applications.