BullyFrame: Cyberbullying Meets FrameNet
Authors: Silvia Brambilla, Alessio Palmero Aprosio, Stefano Menini (Fondazione Bruno Kessler, Università di Bologna, 2019)
Topic: Lexical and Semantic Resources and Analysis

Abstract
This paper presents BullyFrame, a dataset of cyberbulling interactions collected from WhatsApp conversations in Italian and annotated with FrameNet semantic frames. We will describe the creation of the dataset discussing the problematic aspects found in the annotation process, such as the lack of coverage of FrameNet for the annotation of texts extracted from social media. Finally, we present a preliminary study that describes the relations between the frames and the cyberbullying-related annotation of the original dataset.

Introduction
The semantic analysis of a text involves the classification of predicates into a set of events, for which it is important to determine who did what, when and where. For example, in the sentence “In 1912, the Titanic hit an iceberg on its first trip across the Atlantic”, the verb “hit” represents the event, “Titanic” is the main actor of that event, “1912” and “Atlantic” indicate when and where it took place, and so on. The process of extracting the semantic roles and relations in a sentence is called Semantic Role Labeling (SRL), and, in the last years, both resources listing possible events and corpora have been annotated with this kind of information. Examples of such datasets are FrameNet (Ruppenhofer et al., 2006) and PropBank (Palmer et al., 2005). Given the availability of these resources, over the years SRL has gained more attention and has become an important task in computational linguistics, with a growing number of works and evaluations (QasemiZadeh et al., 2019; Basili et al., 2012). Unfortunately, the vast majority of annotated datasets relies mainly on newswire and narrative texts, and their coverage turns out to be inadequate when it comes to annotate more specific domains, such as, for instance, football domain (Torrent et al., 2014) or medicine domain (Tan et al., 2011). Aside from that, over the last decades, ICT technologies and communication habits underwent profound changes, with the greatest part of text production in the world coming from social networks and being usually written in non-standard language. This kind of communication is of fundamental importance, in particular for teenagers’ social life. For instance, according to the last report by the Italian Statistical Institute (ISTAT, 2014) in Italy 82.6 of children aged 11-17 use the mobile phone every day. The use of these new technologies, however, leads also to some undesirable side effects, as the proliferation of hate speech and the digitization of traditional forms of harassment, also known as cyberbullying. Many studies (O’Moore and Kirkham, 2001; Fekkes et al., 2006; Farag et al., 2019) have highlighted that cyberbullying can have a negative impact on the victims’ psychological and emotional well-being and that, in extreme cases, it can lead to self-harm and suicidal thoughts. For this reason, some strategies have been implemented to detect and contrast this phenomenon (Van Hee et al., 2018; Zhao et al., 2016; Menini et al., 2019), but none of them makes use of SRL, and no resources on this topic based on frame semantics have been developed yet. We therefore developed BullyFrame, a dataset annotated with frame semantic annotation, where the messages are taken from a corpus of data on cyberbullying interaction in Italian, gathered through a WhatsApp experimentation with lower secondary school students (Sprugnoli et al., 2018). Our work leads to the release of the annotated corpus (see Section 3), and constitutes a feasibility study, that investigates the potential lacks of FrameNet - resource that does not claim to be exhaustive in its coverage - for the annotation of online chats that, in addition to their non-standard nature, contain offensive language and informal expressions. We show, for instance, that some frames are completely missing, such as those regarding sexual orientation, as discussed in Section 4. In other cases, FrameNet provides frames whose purpose is similar to the needed one, but cannot fit perfectly the meaning of the sentence. For example, the frame “Offenses” refers to acts that violate a legal code, but it is not used for marking offenses (or bad words) between two users, e.g. “idiota” (“idiot” - currently tagged as Mental property), “stronzetta” (“asshole” - left currently with no annotation). Similarly, a sentence like “Ti ricordo che io ho ballato con Kledi” (“I remind you that I danced with Kledi”) cannot be correctly annotated, as neither Evoking, nor Reminder or Remembering * frames are able to capture the meaning of someone who reminds something to another person. In Section 5, we also provide a comparison study to highlight relations between the newly-released frame annotation and the existing one regarding the type of cyberbullying expression. Results show that some of them are strictly connected (even when it is not immediate to understand). Finally, in Section 6 we present Framy, a frame annotation tool that works as a web server and that has been used for annotating BullyFrame.

Conclusion
In this paper, we present and release BullyFrame, an Italian resource consisting of a set of WhatsApp chats with full-text FrameNet annotations. The data, freely accessible on GitHub, increases the availability of resources in Italian. We also discuss how FrameNet lacks certain frames, as it cannot cover some expressions used mainly in social media language. Finally, we describe Framy, a free tool that supports the manual annotation of texts with respect to FrameNet. In the future, we aim to extend this dataset by including other text resources and expanding FrameNet's coverage for the social media domain, addressing informal expressions and emojis.