CSECU _ KDE _ MA at S em E val-2020 Task 8: A Neural Attention Model for Memotion Analysis (2020)

Abstract: A meme is a pictorial representation of an idea or theme. In the age of emerging volume of social media platforms, memes are spreading rapidly from person to person and becoming a trending ways of opinion expression. However, due to the multimodal characteristics of meme contents, detecting and analyzing the underlying emotion of a meme is a formidable task. In this paper, we present our approach for detecting the emotion of a meme defined in the SemEval-2020 Task 8. Our team CSECU_KDE_MA employs an attention-based neural network model to tackle the problem. Upon extracting the text contents from a meme using an optical character reader (OCR), we represent it using the distributed representation of words. Next, we perform the convolution based on multiple kernel sizes to obtain the higher-level feature sequences. The feature sequences are then fed into the attentive time-distributed bidirectional LSTM model to learn the long-term dependencies effectively. Experimental results show that our proposed neural model obtained competitive performance among the participants’ systems.

## introduction nowadays, social media platforms such as facebook, instagram, and twitter become the most popular information sharing medium among the people due to its convenient features and realtime behavior. people usually use the various modalities of information such as textual, visual, and audio to express their views, opinions, breaking news, and ideas here. due to the robust feature of social media, researchers and companies are trying to distill various kinds of information from its contents. but most of the previous studies address only one modality i.e. image information extraction addressed by the computer vision community and textual information extraction by the natural language processing community. however, with the growing ubiquity of internet memes on social media, it is important to extract information from memes. we need to employ a hybrid approach in this regard since a meme comprises of an image with textual information. memotion analysis @xcite is commonly defined as the process of detecting and analyzing the underlying emotion of a meme. it might have a significant impact on addressing various issues related to social media. for example, evil-minded people nowadays use the meme contents to propagate anti-social behavior including online harassment, cyber-bullying, and hate speech. therefore, memotion analysis might help to limit these anti-social behaviors. to address the challenges of memotion analysis on social media contents, @xcite proposed the task 8 at semeval-2020. the task focuses on three related subtasks. task a defines a sentiment classification problem where a system needs to predict whether a meme content is positive, negative, or neutral. whereas task b defines the multilabel multiclass humour classification problem where a system needs to identify the types of humor expressed by a meme. the categories are sarcastic, humorous, offensive, and motivation meme. task c defines a quantification of semantic class problem where a system needs to quantify the extent of each humour class (defined in task b) expressed by a meme. the rest of the paper is structured as follows: section 2 provides a brief overview of prior research. in section 3, we introduce our proposed neural attention model. section 4 includes experiments and evaluations as well as the analysis of our proposed method. some concluded remarks and future directions of our work are described in section 5. early studies on memes focused mainly on automatic meme generation from various sources. among several prominent works, peirson et al. (2018) employed a pre-trained inception-v3 network for image embedding and passed the embedding features to an attention-based deep-layer lstm model for generating meme caption. vyalla et al. (2020) utilized the encoder-decoder based transformer model for meme caption generation. oliveira et al. (2016) designed a rule-based classifier that relied on linguistic triggers for the generation of memes focused on news headlines. besides, some studies shed light on other applications of meme analysis. beskow et al. (2020) proposed a multi-modal deep learning model to identify and characterize the political memes. williams et al. (2016) studied the racial microaggressions and perceptions of internet memes. a few researchers @xcite have tried to distill the inherent sentiment of the meme contents. ## proposed neural attention framework in this section, we describe the details of our proposed neural attention framework. the goal of our proposed approach is to identify several emotion orientations of internet memes including sentiment, humors, and scales of semantic classes. in our proposed architecture, we utilize the ocr (optical character recognition) extracted text contents to identify the emotional orientation of a meme. after extracting meme text, we employ a pre-trained word embedding model to obtain the high-quality distributed vector representations. next, we apply the multi-kernel convolution (mkc) and time-distributed bidirectional long short-term memory (bi-lstm) models to extract the higher-level feature sequences with sequential information from the meme text embeddings. an attention mechanism is employed to amplify the contribution of important elements in the obtained feature representation. the generated output feature sequences are then sent to the fullyconnected prediction module to determine the final category label. next, we describe each component elaborately. ## conclusion in this paper, we presented our approach to the semeval-2020 task 8: memotion analysis. we tackled the problem by employing an attention-based neural network model. though we achieved the competitive performance, there is much room left to improve the performance of our method. we only exploit the information from text content. however, the information extracted from the image also necessary in this context. in the future, we have a plan to address this scenario and introducing several sophisticated deep learning techniques.