Italian Event Detection Goes Deep Learning
Authors: Tommaso Caselli (University of Groningen, 2018)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
This paper reports on a set of English. This paper reports on a set of experiments with different word embeddings to initialize a state-of-the-art BiLSTM-CRF network for event detection and classiﬁcation in Italian, following the EVENTI evaluation exercise. The network obtains a new state-of-the-art result by improving the F1 score for detection of 1.3 points, and of 6.5 points for classiﬁcation, by using a single step approach. The results also provide further evidence that embeddings have a major impact on the performance of such architectures.

Introduction
Current societies are exposed to a continuous ﬂow of information that results in a large production of data (e.g. news articles, micro-blogs, social media posts, among others), at different moments in time. In addition to this, the consumption of information has dramatically changed: more and more people directly access information through social media platforms (e.g. Facebook and Twitter), and are less and less exposed to a diversity of perspectives and opinions. The combination of these factors may easily result in information overload and impenetrable “ﬁlter bubbles”. Events, i.e. things that happen or hold as true in the world, are the basic components of such data stream. Being able to correctly identify and classify them plays a major role to develop robust solutions to deal with the current stream of data (e.g. the storyline framework (Vossen et al., 2015)), as well to improve the performance of many Natural Language Processing (NLP) applications such as automatic summarization and question answering (Q.A.). Event detection and classiﬁcation has seen a growing interest in the NLP community thanks to the availability of annotated corpora (LDC, 2005; Pustejovsky et al., 2003a; O’Gorman et al., 2016; Cybulska and Vossen, 2014) and evaluation campaigns (Verhagen et al., 2007; Verhagen et al., 2010; UzZaman et al., 2013; Bethard et al., 2015; Bethard et al., 2016; Minard et al., 2015). In the context of the 2014 EVALITA Workshop, the EVENTI evaluation exercise (Caselli et al., 2014)1 was organized to promote research in Italian Temporal Processing, of which event detection and classiﬁcation is a core subtask. Since the EVENTI campaign, there has been a lack of further research, especially in the application of deep learning models to this task in Italian. The contributions of this paper are the followings: i.) the adaptation of a state-of-the-art sequence to sequence (seq2seq) neural system to event detection and classiﬁcation for Italian in a single step approach; ii.) an investigation on the quality of existing Italian word embeddings for this task; iii.) a comparison against a state-of-the-art discrete classiﬁer. The pre-trained models and scripts running the system (or re-train it) are publicly available.

Conclusion
This paper has investigated the application of different word embeddings for the initialization of a state-of-the-art Bi-LSTM-CRF network to 14Note that POS are manually tagged for events, not for their components. solve the event detection and classiﬁcation task in Italian, according to the EVENTI exercise. We obtained new state-of-the-art results using the Fastext-It embeddings, and improved the F1-class score of 6.5 points in strict evaluation mode. As for the event detection subtask, we observe a limited improvement (+1.3 points in strict F1), mainly due to gains in Recall. Such results are extremely positive as the task has been modeled in a single step approach, i.e. detection and classiﬁcation at once, for the ﬁrst time in Italian. Further support that embeddings have a major impact in the performance of neural architectures is provided, as the variations in performance of the Bi-LSMTCRF models show. This is due to a combination of factors such as dimensionality, (raw) data, and the method used for generating the embeddings. Future work should focus on the development of embeddings that move away from the basic word level, integrating extra layers of linguistic analysis (e.g. syntactic dependencies) (Komninos and Manandhar, 2016), that have proven to be very powerful for the same task in English.