Abstract: Figurative Language (FL) seems ubiquitous in all social-media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of Natural Language Processing (NLP), mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper we employ advanced Deep Learning (DL) methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work [71], we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which, is further enhanced with the employment and devise of a recurrent convolutional neural network (RCNN). With this set-up, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state of the art methodologies and systems. Results demonstrate that the proposed methodology achieves state of the art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.

Introduction: In the networked-world era the production of (structured or unstructured) data is increasing with most of our knowledge being created and communicated via web-based social channels BIBREF1. Such data explosion raises the need for efficient and reliable solutions for the management, analysis and interpretation of huge data sizes. Analyzing and extracting knowledge from massive data collections is not only a big issue per-se, but also challenges the data analytics state-of-the-art BIBREF2, with statistical and machine learning methodologies paving the way, and deep learning (DL) taking over and presenting highly accurate solutions BIBREF3. Relevant applications in the field of social media cover a wide spectrum, from the categorization of major disasters BIBREF4 and the identification of suggestions BIBREF5 to inducing users’ appeal to political parties BIBREF6. The raising of computational social science BIBREF7, and mainly its social media dimension BIBREF8, challenge contemporary computational linguistics and text-analytics endeavors. The challenge concerns the advancement of text analytics methodologies towards the transformation of unstructured excerpts into some kind of structured data via the identification of special passage characteristics, such as its emotional content (e.g., anger, joy, sadness) BIBREF9. In this context, Sentiment Analysis (SA) comes into play, targeting the devise and development of efficient algorithmic processes for the automatic extraction of a writer’s sentiment or emotion as conveyed in text excerpts. Relevant efforts focus on tracking the sentiment polarity of single utterances, which in most cases is loaded with a lot of subjectivity and a degree of vagueness BIBREF10. Contemporary research in the field utilizes data from social media resources (e.g., Facebook, Twitter) as well as other short text references in blogs, forums etc BIBREF11. However, users of social media tend to violate common grammar and vocabulary rules and even use various figurative language forms to communicate their message. In such situations, the sentiment inclination underlying the literal content of the conveyed concept may significantly differ from its figurative context, making SA tasks even more puzzling. Evidently, single turn text lack in detecting sentiment polarity on sarcastic and ironic expressions, as already signified in the relevant “SemEval-2014 Sentiment Analysis task 9” BIBREF12. Moreover, lacking of facial expressions and voice tone require context aware approaches to tackle such a challenging task and overcome its ambiguities BIBREF13. As sentiment is the emotion behind customer engagement, SA finds its realization in automated customer aware services, elaborating over user’s emotional intensities BIBREF14. Most of the related studies utilize single turn texts from topic specific sources, such as Twitter, Amazon, IMDB etc. Hand crafted and sentiment-oriented features, indicative of emotion polarity, are utilized to represent respective excerpt cases. The formed data are then fed traditional machine learning classifiers (e.g. SVM, Random Forest, multilayer perceptrons) or DL techniques and respective complex neural architectures, in order to induce analytical models that are able to capture the underlying sentiment content and polarity of passages BIBREF15, BIBREF16, BIBREF17. The linguistic phenomenon of figurative language (FL) refers to the contradiction between the literal and the non-literal meaning of an utterance BIBREF18. Literal written language assigns ‘exact’ (or ‘real’) meaning to the used words (or phrases) without any reference to putative speech figures. In contrast, FL schemas exploit non-literal mentions that deviate from the exact concept presented by the used words and phrases. FL is rich of various linguistic phenomena like ‘metonymy’ reference to an entity stands for another of the same domain, a more general case of ‘synonymy’; and ‘metaphors’ systematic interchange between entities from different abstract domains BIBREF19. Besides the philosophical considerations, theories and debates about the exact nature of FL, findings from the neuroscience research domain present clear evidence on the presence of differentiating FL processing patterns in the human brain BIBREF20, BIBREF21, BIBREF22, BIBREF23, BIBREF14, even for woman-man attraction situations! BIBREF24. A fact that makes FL processing even more challenging and difficult to tackle. Indeed, this is the case of pragmatic FL phenomena like irony and sarcasm that main intention of in most of the cases, are characterized by an oppositeness to the literal language context. It is crucial to distinguish between the literal meaning of an expression considered as a whole from its constituents’ words and phrases. As literal meaning is assumed to be invariant in all context at least in its classical conceptualization BIBREF25, it is exactly this separation of an expression from its context that permits and opens the road to computational approaches in detecting and characterizing FL utterance. We may identify three common FL expression forms namely, irony, sarcasm and metaphor. In this paper, figurative expressions, and especially ironic or sarcastic ones, are considered as a way of indirect denial. From this point of view, the interpretation and ultimately identification of the indirect meaning involved in a passage does not entail the cancellation of the indirectly rejected message and its replacement with the intentionally implied message (as advocated in BIBREF26, BIBREF27). On the contrary ironic/sarcastic expressions presupposes the processing of both the indirectly rejected and the implied message so that the difference between them can be identified. This view differs from the assumption that irony and sarcasm involve only one interpretation BIBREF28, BIBREF29. Holding that irony activates both grammatical / explicit as well as ironic / involved notions provides that irony will be more difficult to grasp than a non-ironic use of the same expression. Despite that all forms of FL are well studied linguistic phenomena BIBREF28, computational approaches fail to identify the polarity of them within a text. The influence of FL in sentiment classification emerged both on SemEval-2014 Sentiment Analysis task BIBREF12 and BIBREF19. Results show that Natural Language Processing (NLP) systems effective in most other tasks see their performance drop when dealing with figurative forms of language. Thus, methods capable of detecting, separating and classifying forms of FL would be valuable building blocks for a system that could ultimately provide a full-spectrum sentiment analysis of natural language. In literature we encounter some major drawbacks of previous studies and we aim to resolve with our proposed method: Many studies tackle figurative language by utilizing a wide range of engineered features (e.g. lexical and sentiment based features) BIBREF30, BIBREF31, BIBREF0, BIBREF32, BIBREF33, BIBREF34 making classification frameworks not feasible. Several approaches search words on large dictionaries which demand large computational times and can be considered as impractical BIBREF0, BIBREF34 Many studies exhausting preprocess the input texts, including stemming, tagging, emoji processing etc. that tend to be time consuming especially in large datasets BIBREF35, BIBREF36. Many approaches attempt to create datasets using social media API’s to automatically collect data rather than exploiting their system on benchmark datasets, with proven quality. To this end, it is impossible to be compared and evaluated BIBREF35, BIBREF37, BIBREF36. To tackle the aforementioned problems, we propose an end-to-end methodology containing none hand crafted engineered features or lexicon dictionaries, a preprocessing step that includes only de-capitalization and we evaluate our system on several benchmark dataset. To the best of our knowledge, this is the first time that an unsupervised pre-trained Transformer method is used to capture figurative language in many of its forms. The rest of the paper is structured as follows, in Section SECREF2 we present the related work on the field of FL detection, in Section SECREF3 we present our proposed method along with several state-of-the-art models that achieve high performance in a wide range of NLP tasks which will be used to compare performance, the results of our experiments are presented in Section SECREF4, and finally our conclusion is in Section SECREF5.

Question: How are the three different forms defined in this work?
Answer: Irony, sarcasm and metaphor are figurative language form. Irony and sarcasm are considered as a way of indirect denial.