IIIT _ DWD @ EACL 2021: Identifying Troll Meme in T amil using a hybrid deep learning approach (2021)

Abstract: Social media are an open forum that allows people to share their knowledge, abilities, talents, ideas, or expressions. Simultaneously, it also allows people to post disrespectful, trolling, defamation, or negative content targeting users or the community based on their gender, race, religious beliefs, etc. Such posts are available in the form of text, image, video, and meme. Among them, memes are currently widely used to disseminate offensive material amongst people. It is primarily in the form of pictures and text. In the present paper, troll memes are identified, which is necessary to create a healthy society. To do so, a hybrid deep learning model combining convolutional neural networks and bidirectional long short term memory is proposed to identify trolled memes. The dataset used in the study is a part of the competition EACL 2021: Troll Meme classification in Tamil. The proposed model obtained 10th rank in the competition and reported a precision of 0.52, recall 0.59, and weighted F10.3.

## introduction social media has become one of the essential components of our life, thanks to digitization. over the years, the number of social media users has grown at an exponential pace. according to the survey of wearesocial foot_0 , out of around 7.8 billion people worldwide, 3.8 billion people are active social media users @xcite . they participate in many ways on social media, like posting comments (in the form of text, image, video, audio), online chatting (text, audio, and video ), hosting live events, and so on @xcite . the information available in these posts is overwhelming and very difficult to manage for users. therefore, people always look for summarized information available on social media instead of reading the information in the form of text paragraphs. internet memes are one such summarized version of posts that are attractive, colorful, and extensively being used among the social media community. an internet meme is a viral concept that alters as it spreads out foot_1 . it is usually taken as a movie clip or scene with an added subtext that has a modified meaning. as memes contain both image and text information it becomes easier to understand the context. memes are being used to spread news, sharing knowledge, emotion, ideas, talents, etc. it is also being used by government agencies and industry professionals to disseminate awareness programs, advertise their products, ideas, and so on. at the same time, memes are also extensively being used to spread wrong, hate, or offensive information targeting a particular community based on their color, appearance, gender, religious believes, and so on @xcite @xcite . such offensive memes that intend to insult individuals or groups are termed as troll memes. trolling memes can create social media conflicts and influence certain events. it can cause loss of business or ruin somebody's life. therefore, it is essential to identify such trolling memes and restrict their spread. there is a number of works proposed in the literature for memes analysis. for example, @xcite , proposed a study on the themes contained in the text of memes extracted from twitter. @xcite performed experiments using a neural network model. they found that thirty percent of memes in their dataset had a political theme and were mostly shared by democrats than repub-licans. @xcite presented a discourse analysis of memes where they found that to understand memes, knowing social content of it is necessary, along with the text and picture. they also concluded that memes can be used for multiple purposes such as humour, sarcasm, identifying feelings, and so on. @xcite presented a memotion analysis dataset for categorizing memes into annotated tags namely sentiment, and type of humor that is, sarcastic, humorous, or offensive. @xcite proposed a hybrid approach using inception network and long short term memory network for extracting features from image and text respectively to identify hate content in the multimodal dataset. @xcite proposed a multimodal meme dataset (mul-tioff) for offensive content detection. they used an early fusion technique to combine the image and text modality. as discussed above, most of the existing works have proposed a hybrid approach for tackling image and text present in memes. moreover, the text language used in those memes was english. the current paper is unique in the sense that it identifies offensive or trolling memes where texts are in the tamil language. tamil was the first to be categorized as india's classical language and is one of the world's longest-surviving classical languages. over 55 percent of the epigraphic inscriptions discovered by the archaeological survey of india (about 55,000) are in the tamil language. in sri lanka and on commercial products in thailand and egypt, tamil language inscriptions written in brahmi script have been found @xcite @xcite . the dataset used in the paper is a part of the task eacl 2021:meme classification for tamil. the problem proposed is a binary classification problem where every meme has to be categorized into tags troll and not troll. the classification model used for this was a hybrid model having 2-layered cnn and bi-lstm in parallel foot_2 . the proposed model on test data reported precision 0.52, recall 0.59, and weighted f1 0.3. the rest of the article is organized as follows; section 2 discusses the given task and dimension of the dataset. the methodology of the paper is explained in section 3. this is followed by exper-imental results which are explained in section 4. finally, section 5 concludes the paper. ## task and data description as per the best of our knowledge, this is the first task of identifying offensive troll meme in tamil. the task was proposed in eacl 2021. the objective of the task was to classify each meme into two categories troll and not troll @xcite . training and testing dataset of the task was released in a phased manner where initially, training data was released in a folder containing two files; a csv and an image folder. the image folder contained 2300 images out of that 1282 image names were starting with troll and other 1080 image names were starting with not troll. based on their naming convention we identified which images belonged to troll class or not troll class. the csv file contained two fields, image names and their captions (or texts). similarly, for the test dataset, two files (image folder and csv file) were given to predict the final class label. but, in the test set the image names were not having class information. the total test set count was 667. ## methodology to identify troll meme in tamil a hybrid binary classification model is proposed. the detailed flow of the proposed model is shown in figure 1 . the detailed working of the model is discussed in the subsections. ## result all models were developed in python using libraries sklearn, keras, pandas, numpy, and so on. we experimented with different variations of the hybrid settings such as a parallel single-layer cnn-lstm model, a parallel single layer cnn-cnn model, a parallel single-layer cnn-bi-lstm model, a parallel multilayer cnn-lstm model, a parallel multilayer cnn-cnn model, and a par-allel multi cnn layer and single bi-lstm layer model. we are reporting here the results of the best model, a parallel multi cnn layer and a single bi-lstm layer model for the test dataset. initially, organizers provided test data without labels. the prediction of each test data was submitted to the competitions. later, the organizers (eacl 2021: troll meme classification in tamil) released the score of the testing data as 0.52 precision, 0.59 recall, and 0.3 f1 scores. our model obtained the 10th position in the ranking released by organizers. the 1st ranked model in the competition obtained a precision of 0.57, recall 0.6, and f1-score 0.55. as it is observed, the recall of the current model (0.59) was very close to the recall of the 1st ranked model (0.60). however, the precision of the submitted model (0.52) was a little far from the precision of the best-ranked model (0.59). that means the false positive rate of our model was higher. ## conclusion and future work the current paper identified troll memes in social media platform using a parallel cnn-bi-lstm based network. the task was proposed by eacl 2021 for troll meme classification in tamil. the proposed model categorised each meme into one of the classes troll and not troll. the model reported the precision of 0.52, recall 0.59, and f1-score 0.3 on the test dataset. our model ranked at 10th position in the competition. the best model in the competition reported 0.57 precision, 0.6 recall, and 0.55 f1-score. that means a lot of works has to be done to improve the accuracy of the model. as future work, several transfer learning models can be employed such as vgg-16, resnet50, inception network, and so on for image and bert classifiers with a few pre-trained embedding models like glove, word2vec, fasttext, and so on for text processing.