Irony Detection: from the Twittersphere to the News Space
Authors: Alessandra Cervone, Evgeny A. Stepanov, Fabio Celli, Giuseppe Riccardi (Università di Trento, 2017)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
Automatic detection of irony is one of the hot topics for sentiment analysis, as it changes the polarity of text. Most of the work has been focused on the detection of figurative language in Twitter data due to relative ease of obtaining annotated data, thanks to the use of hashtags to signal irony. However, irony is present generally in natural language conversations and in particular in online public fora. In this paper, we present a comparative evaluation of irony detection from Italian news fora and Twitter posts. Since irony is not a very frequent phenomenon, its automatic detection suffers from data imbalance and feature sparseness problems. We experiment with different representations of text -- bag-of-words, writing style, and word embeddings to address the feature sparseness; and balancing techniques to address the data imbalance.

Introduction
The detection of irony in user generated content is one of the major issues in sentiment analysis and opinion mining AUTHOR. The problem is that irony can flip the polarity of apparently positive sentences, negatively affecting the performance of sentiment polarity classification AUTHOR. Detecting irony from text is extremely difficult because it is deeply related to many out-of-text factors such as context, intonation, speakers' intentions, background knowledge and so on. This also affects interpretation and annotation of irony by humans, often leading to low inter-annotator agreements. Twitter posts are frequently used for the irony detection research, since users often signal irony in their posts utilizing hashtags such as \#irony, \#justjoking, etc. Despite the relative ease of collecting the data, Twitter is a very particular kind of text. In this paper we experiment with different representations of text to evaluate the utility of Twitter data for the detection of irony in text coming from other sources such as news fora. The representations of text -- bag-of-words, writing style, and word embeddings -- are chosen such that they are not dependent on the resources available for the language. Due to the fact that irony is less frequent than literal meaning, the data is usually imbalanced. We experiment with balancing techniques such as random undersampling, random oversampling and cost-sensitive training to observe its effects on a supervised irony detection. The paper is structured as follows. In Section sec:works we introduce related work on irony. In Section sec:data we describe the corpora used throughout experiments. In Sections sec:method and sec:exp we describe the methodology and the result of the experiments. In Section sec:conc we provide concluding remarks.

Conclusion
We have presented experiments on irony detection in Italian Twitter and news fora data comparing different document representations -- bag-of-words, writing style as stylometric features, and word embeddings. The objective is to evaluate the suitability of Twitter data for detecting irony in news fora. The models were compared for balanced and imbalanced training, as well as cross-corpora performance. We have observed that the bag-of-words representation with imbalanced cost-insensitive training produces the best results (micro-F_1) across settings, closely followed by cost-sensitive training. The models outperform the results on irony detection in Italian tweets AUTHOR in terms of macro-F_1 scores reported for SENTIPOLC 2016 AUTHOR. However, micro-F_1 is the most informative metric for the downstream application of irony detection, as it considers the total amount of true positives. Given that the highest micro-F_1 is attained by the majority baselines for both corpora (0.8829 for SENTIPOLC and 0.7996 for CorEA), the task of irony detection is far from being solved.