A Multi-Modal Method for Satire Detection using Textual and Visual Cues (2020)

Abstract: Satire is a form of humorous critique, but it is sometimes misinterpreted by readers as legitimate news, which can lead to harmful consequences. We observe that the images used in satirical news articles often contain absurd or ridiculous content and that image manipulation is used to create fictional scenarios. While previous work have studied text-based methods, in this work we propose a multi-modal approach based on state-of-the-art visiolinguistic model ViLBERT. To this end, we create a new dataset consisting of images and headlines of regular and satirical news for the task of satire detection. We fine-tune ViLBERT on the dataset and train a convolutional neural network that uses an image forensics technique. Evaluation on the dataset shows that our proposed multi-modal approach outperforms image-only, text-only, and simple fusion baselines.

## introduction satire is a literary device that writers employ to mock or ridicule a person, group, or ideology by passing judgment on them for a cultural transgression or poor social behavior. satirical news utilizes humor and irony by placing the target of the criticism into a ridiculous, fictional situation that the reader must suspend their disbelief and go along with @xcite . however, despite what absurd content satirical news may contain, it is often mistaken by readers as real, legitimate news, which may then lead to the unintentional spread of misinformation. in a recent survey conducted by the conversation @xcite , up to 28% of republican respondents and 14% of democratic respondents reported that they believed stories fabricated by the babylon bee, a satirical news website, to be "definitely true". in these instances, the consequences of satire are indistinguishable from those of fake news. to reduce the spread of misinformation, social media platforms have partnered with third-party factcheckers to flag false news articles and tag articles from known satirical websites as satire for users @xcite . however, due to the high cost and relative inefficiency of employing experts to manually annotate articles, many researchers have tackled the challenge of automated satire detection. existing models for satirical news detection have yet to explore the visual domain of satire, even though image thumbnails of news articles may convey information that reveals or disproves the satirical nature of the articles. in the field of cognitive-linguistics, @xcite observed the use of altered images showing imaginary scenarios on the satirical news show the daily show. this phenomenon also extends to satirical news articles, as seen in figure 1 . for example, figure 1(a ) depicts the marvel cinematic universe character hulk from the film avengers: infinity war and the united states president donald trump spliced together. alone, each of the two images is serious and not satirical, but, since they come from drastically different contexts, combining the two images creates a clearly ridiculous thumbnail that complements the headline of the article. in our work, we propose a multi-modal method for detecting satirical news articles. we hypothesize that 1) the content of news thumbnail images when combined with text, and 2) detecting the presence of manipulated or added characters and objects, can aid in the identification of satirical articles. 2 related work previous work proposed methods for satirical news detection using textual content @xcite . some works utilize classical machine learning algorithms such as svm with handcrafted features from factual and satirical news headlines and body text, including bag-of-words, n-grams, and lexical features @xcite . more recent works use deep learning to extract learned features for satire detection. @xcite proposed a hierarchical model with attention mechanism and handcrafted linguistic features to understand satire at a paragraph and article-level. while previous work utilize visiolinguistic data for similar tasks, there is no related work that employs multi-modal data to classify articles into satirical and factual news. @xcite created a dataset containing images and text for fake news detection in posts on the social media website reddit. while they include a category for satire/parody in their 6-way dataset, since they use only content that has been submitted by reddit users, it is not representative of mainstream news media. multi-modal approaches have also been tried in sarcasm detection; @xcite compiled a dataset of scenes from popular tv shows and @xcite used tweets comprising of text and images from twitter. ## conclusion and future investigations in this paper, we create a multi-modal satire detection dataset and propose two models for the task based on the characteristics of satirical images and their relationships with the headlines. while our model based on image tampering detection performed significantly worse than the baselines, empirical evaluation showed the efficacy of our proposed multi-modal approach compared to simple fusion and uni-modal models. in future work on satire detection, we will incorporate image forensics methods to identify image splicing in satirical images, body text of articles instead of just headlines, as well as knowledge about politics and other current issues.