Question: What is an example of a difficult-to-classify case?
Evidence: In order to provide directions for future work, we analyze the errors made by the classifier trained on the extended features on the four prediction tasks.
Errors on Intention (I) prediction: The lack of background is a major problem when identifying trolling comments. For example, “your comments fit well in Stormfront” seems inoffensive on the surface. However, people who know that Stormfront is a white supremacist website will realize that the author of this comment had an annoying or malicious intention. But our system had no knowledge about it and simply predicted it as non-trolling. These kind of errors reduces recall on the prediction of trolling comments. A solution would be to include additional knowledge from anthologies along with a sentiment or polarity. One could modify NELL BIBREF12 to broaden the understanding of entities in the comments.
Non-cursing aggressions and insults This is a challenging problem, since the majority of abusive and insulting comments rely on profanity and swearing. The problem arises with subtler aggressions and insults that are equally or even more annoying, such as “Troll? How cute.” and “settle down drama queen”. The classifier has a more difficult task of determining that these are indeed aggressions or insults. This error also decreases the recall of trolling intention. A solution would be to exploit all the comments made by the suspected troll in the entire conversation in order to increase the chances of finding curse words or other cues that lead the classifier to correctly classify the comment as trolling.
Another source of error is the presence of controversial topic words such as “black”,“feminism”, “killing”, “racism”, “brown”, etc. that are commonly used by trolls. The classifier seems too confident to classify a comment as trolling in the presence of these words, but in many cases they do not. In order to ameliorate this problem, one could create ad-hoc word embeddings by training glove or other type of distributed representation on a large corpus for the specific social media platform in consideration. From these vectors one could expect a better representation of controversial topics and their interactions with other words so they might help to reduce these errors.
Errors on Disclosure (D) prediction: A major source of error that affects disclosure is the shallow meaning representation obtained from the BOW model even when augmented with the distributional features given by the glove vectors. For example, the suspected troll's comment “how to deal with refugees? How about a bullet to the head” is clearly mean-spirited and is an example of disclosed trolling. However, to reach that conclusion the reader need to infer the meaning of “bullet to the head” and that this action is desirable for a vulnerable group like migrants or refugees. This problem produces low recall for the disclosed prediction task. A solution for this problem may be the use of deeper semantics, where we represent the comments and sentences in their logical form and infer from them the intended meaning.
Errors on Interpretation (R) prediction: it is a common practice from many users to directly ask the suspected troll if he/she is trolling or not. There are several variations of this question, such as “Are you a troll?” and “not sure if trolling or not”. While the presence of a question like these seems to give us a hint of the responder's interpretation, we cannot be sure of his interpretation without also considering the context. One way to improve interpretation is to exploit the response strategy, but the response strategy in our model is predicted independently of interpretation. So one solution could be similar to the one proposed above for the disclosure task problem: jointly learning classifiers that predict both variables simultaneously. Another possibility is to use the temporal sequence of response comments and make use of older response interpretation as input features for later comments. This could be useful since commenters seem to influence each other as they read through the conversation.
Errors on Response Strategy (B) prediction: In some cases there is a blurry line between “Frustrate” and “Neutralize”. The key distinction between them is that there exists some criticism in the Frustrate responses towards the suspected troll's comment, while “Neutralizing” comments acknowledge that the suspected troll has trolling intentions, but gives no importance to them. For example, response comments such as “oh, you are a troll” and “you are just a lame troll” are examples of this subtle difference. The first is a case of “neutralize” while the second is indeed criticizing the suspected troll's comment and therefore a “frustrate” response strategy. This kind of error affects both precision and recall for these two classes. A possible solution could be to train a specialized classifier to disambiguate between “frustrate” and “neutralize” only.
Another challenging problem is the distinction between the classes “Troll” and “Engage”. This is true when the direct responder is intensely flared up with the suspected comment to the point that his own comment becomes a trolling attempt. A useful indicator for distinguishing these cases are the presence of insults, and to detect them we look for swear words, but as we noted before, there is no guarantee that swear words are used for insulting. This kind of error affects the precision and recall for the “troll” and “engage” classes. A solution to this problem may be the inclusion of longer parts of the conversation. It is typical in a troll-engaged comment scheme to observe longer than usual exchanges between two users, and the comments evolve in very agitated remarks. One may then use this information to disambiguate between the two classes.
In order to provide directions for future work, we analyze the errors made by the classifier trained on the extended features on the four prediction tasks.

Errors on Intention (I) prediction: The lack of background is a major problem when identifying trolling comments.
Non-cursing aggressions and insults This is a challenging problem, since the majority of abusive and insulting comments rely on profanity and swearing. 
Another source of error is the presence of controversial topic words such as “black”,“feminism”, “killing”, “racism”, “brown”, etc. that are commonly used by trolls.
Errors on Disclosure (D) prediction: A major source of error that affects disclosure is the shallow meaning representation obtained from the BOW model even when augmented with the distributional features given by the glove vectors.
Errors on Interpretation (R) prediction: it is a common practice from many users to directly ask the suspected troll if he/she is trolling or not. 
Errors on Response Strategy (B) prediction: In some cases there is a blurry line between “Frustrate” and “Neutralize”. 
Another challenging problem is the distinction between the classes “Troll” and “Engage”. 
Answer: The answers are shown as follows:
* The lack of background
* Non-cursing aggressions and insults
* the presence of controversial topic words 
*  shallow meaning representation
* directly ask the suspected troll if he/she is trolling or not
* a blurry line between “Frustrate” and “Neutralize”
* distinction between the classes “Troll” and “Engage”