Question: What are the two levels of annotation
Evidence: To operationalise the task of automatic cyberbullying detection, we developed and tested a fine-grained annotation scheme and applied it to our corpora. While a detailed overview of the guidelines is presented in our technical report BIBREF30 , we briefly present the categories and main annotation steps below.
Threat/Blackmail: expressions containing physical or psychological threats or indications of blackmail.
Insult: expressions meant to hurt or offend the victim.
General insult: general expressions containing abusive, degrading or offensive language that are meant to insult the addressee.
Attacking relatives: insulting expressions towards relatives or friends of the victim.
Discrimination: expressions of unjust or prejudicial treatment of the victim. Two types of discrimination are distinguished (i.e., sexism and racism). Other forms of discrimination should be categorised as general insults.
Curse/Exclusion: expressions of a wish that some form of adversity or misfortune will befall the victim and expressions that exclude the victim from a conversation or a social group.
Defamation: expressions that reveal confident or defamatory information about the victim to a large public.
Sexual Talk: expressions with a sexual meaning or connotation. A distinction is made between innocent sexual talk and sexual harassment.
Defense: expressions in support of the victim, expressed by the victim himself or by a bystander.
Bystander defense: expressions by which a bystander shows support for the victim or discourages the harasser from continuing his actions.
Victim defense: assertive or powerless reactions from the victim.
Encouragement to the harasser: expressions in support of the harasser.
Other: expressions that contain any other form of cyberbullying-related behaviour than the ones described here.
Based on the literature on role-allocation in cyberbullying episodes BIBREF51 , BIBREF50 , four roles are distinguished, including victim, bully, and two types of bystanders.
Harasser or Bully: person who initiates the bullying.
Victim: person who is harassed.
Bystander-defender: person who helps the victim and discourages the harasser from continuing his actions.
Bystander-assistant: person who does not initiate, but helps or encourages the harasser.
Essentially, the annotation scheme describes two levels of annotation. Firstly, the annotators were asked to indicate, at the post level, whether the post under investigation was related to cyberbullying. If the post was considered a signal of cyberbullying, annotators identified the author's role. Secondly, at the subsentence level, the annotators were tasked with the identification of a number of fine-grained text categories related to cyberbullying. More concretely, they identified all text spans corresponding to one of the categories described in the annotation scheme. To provide the annotators with some context, all posts were presented within their original conversation when possible. All annotations were done using the Brat rapid annotation tool BIBREF52 , some examples of which are presented in Table TABREF33 .
font=footnotesize,sc,justification=centering,labelsep=period
Firstly, the annotators were asked to indicate, at the post level, whether the post under investigation was related to cyberbullying. If the post was considered a signal of cyberbullying, annotators identified the author’s role. Secondly, at the subsentence level
Answer: The indicate and subsentence level.