YNU - HPCC at S em E val-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis (2020)

Abstract: this paper proposed a parallel-channel model to process the textual and visual information in memes and then analyze the sentiment polarity of memes. In the shared task of identifying and categorizing memes, we preprocess the dataset according to the language behaviors on social media. Then, we adapt and fine-tune the Bidirectional Encoder Representations from Transformers (BERT), and two types of convolutional neural network models (CNNs) were used to extract the features from the pictures. We applied an ensemble model that combined the BiLSTM, BIGRU, and Attention models to perform cross domain suggestion mining. The officially released results show that our system performs better than the baseline algorithm

## introduction in recent years, memes that combine pictures and text have been widely used in social media. using memes can help users to express richer meaning and emotion compared with using text or images alone; hence, it is worthwhile to analyze the sentiment expressions of memes.moreover, recognizing and analyzing the meaning and sentiment of memes is much more difficult than analyzing social texts or pictures. in semeval-2020 task 8:memotion analysis @xcite , the organizers hoped that the task would increase the research attention given to the topic. the task is divided into three subtasks. memes and this issue have attracted the attention of researchers. in a previous study, borth (2013) pioneered the sentiment analysis of visual content with sentibank. another study implemented optical character recognition (ocr) to extract the text captions of memes and then classified the sentiment polarity of the text using the naive bayes algorithm @xcite . for a similar meme sentiment analysis task, @xcite developed a multimodal sentiment analysis method for image-text posts, and their experiments showed that this method achieves excellent performance on the flickr benchmark dataset. @xcite used glove to map the text to a high dimensional space and fine-tuned the pictures through inception (a pretrained deep convolutional neural network). in this paper, we propose a parallel channel model that includes a text channel, which is implemented to process the text in memes, and an image channel for image analysis. the text channel implements the bilstm, bigru, and bilstm with attention models. for the image channel, a multilayer cnn model and resnet152 @xcite were applied to capture the image features. then, the information in the two modalities is combined by a dense layer after concatenation. the experimental results show that our approach achieved good performance. the remainder of this paper is organized as follows. section 2 describes the proposed parallel channel model, and section 3 presents the implementation details and experimental results. the conclusions of this study are presented in section 4. ## experiments and evaluation in this section, experiments were conducted to evaluate the proposed models on both subtasks. we also report the results of the official review. the details of the experiment are described as follows. ## conclusion in this paper, we describe a task system that we submitted to semeval-2020 for memotion analysis. we propose a two parallel channel model. in the text channel, we use 3 rnn models and 2 types of pretraining vectors. in the image channel, we used a pretrained model and a cnn model. we participated in subtasks a and b, and obtained nineteenth place in subtask a. in future work, we will test more novel fusion methods so that the picture features can be better combined with token embedding.