SSNCSE _ NLP @ D ravidian L ang T ech- EACL 2021: Meme classification for T amil using machine learning approach (2021)

Abstract: Social media are interactive platforms that facilitate the creation or sharing of information, ideas or other forms of expression among people. This exchange is not free from offensive, trolling or malicious contents targeting users or communities. One way of trolling is by making memes. A meme is an image or video that represents the thoughts and feelings of a specific audience. The challenge of dealing with memes is that they are region-specific and their meaning is often obscured in humour or sarcasm. A meme is a form of media that spreads an idea or emotion across the internet. The multi modal nature of memes, postings of hateful memes or related events like trolling, cyberbullying are increasing day by day. Memes make it even more challenging since they express humour and sarcasm in an implicit way, because of which the meme may not be offensive if we only consider the text or the image. In this paper we proposed a approach for meme classification for Tamil language that considers only the text present in the meme. This work explains the submissions made by SSNCSE NLP in DravidanLangTechEACL2021 task for meme classification in Tamil language. We achieve F1 scores of 0.50 using the proposed approach using the test-set.

## introduction more recently, as memes and gifs dominate the social feeds; typo-graphic visual content has become a considerable element of social media. 'meme' is a viral image or video often altered by internet users for humorous effect. these convey human expressions but with a wide range of emotions, and often require context to fully understand humor and sarcasm. "sarcasm is a type of sentiment where people express their negative feelings using positive or intensified positive words in the text @xcite ". the meme classification is handled by considering both the linguistic features as well as visual features. this task will be considered as multi modal approach. the approaches of natural language processing and computer vision are to be used for the meme classification task. using this idea, a model has to be designed which deals with the challenges associated with the semantic nature of text and image, and performs classification on the emotional responses obtained. tamil language is official language of tamil nadu, singapore, and sri lanka @xcite . from the 6th century bce to the 3rd century ce, the sangam (koodal) period encompassed ancient tamil nadu, kerala, and portions of sri lanka (then known as tamilakam). it was named after the illustrious madurai-based sangam (koodal) academies of poets and scholars. tamil was the first indian classical language to be listed as classical language, and it is one of the world's oldest classical languages still in use by common people. inscriptions in the tamil language written in tamili (tamil-bharmi) script have been found in sri lanka, thailand, and egypt. this work explains the submissions made by ssncse nlp in dravidanlangtech-eacl2021 task for meme classification in tamil language. the remainder of the paper is organized as follows. section 2 discusses the related work on meme classification and related tasks using text and images. the dataset about the shared task is described in section . section 4 outlines the features and machine learning algorithms used for this task. results are discussed in section 5. section 6 concludes the paper. ## related work researchers experimented with few approaches used in the text sentiment classification task as well as emotion classification task using images. sentiment analysis from the text is essential in understanding the idea behind the meme. the authors of (wang and wen, 2015) concentrate on generating meme descriptions through non parametric estimation approaches which uses lexical features, pos features, named-entity features, frame-semantics features and dependency triples. in @xcite , the memes are classified by applying image processing and ocr tesseract which are combined with naïve bayes algorithm. the survey of using visual linguistic information used for meme classification, image capationed are discussed in @xcite . the multioff multimodal meme data set for offensive content detection data set was developed by @xcite ) @xcite . most of the research work on meme classification uses both linguistic and visual features for better performance. in our proposed approach, only text based features were used. the proposed work have been described in the next section. ## data-set analysis the data set given by the shared task organizers contains the image file and transcriptions present in the image file. all the image files are in jpeg format and have text embedded on them in the tamil language @xcite , @xcite . later, each meme has been annotated by three annotators (native speakers) in troll or not troll class. the class label (troll or not troll) of the data is embedded in the filename. e.g. a meme with filename "troll 0.jpg" has a label "troll", similarly, "not troll 0.jpg" has label not troll. further, the transcription of captions for each file is given in the data set. the data distribution of train and test set is given in ## experimental setup and features from the given text transcription, n-gram and tfidf and bert embeddings are extracted as features. as the content of the memes is a mix of dravidian language grammar in roman lexicons along with english grammar, it becomes challenging to find pre-trained models for this context. so a simple n-gram approach is considered. also, the advancements done by the transformer model for pretraining and the availability of multilingual trained models encourage to experiment with bert pretrained embeddings. the extracted features are used to train machine learning models such as multi-layer perceptron, random forest, k-nearest neighbour, and the performance of these models are compared. for shared task for meme classification is provided only with training set. to analyze the performance of the proposed approach, 5 fold cross validation is used. the metrics such as accuracy and weighted average f1 score are analyzed. the machine learning model implementations and the metrics of comparison is used from scikit-learn foot_0 . ## observations the output from n-gram embedding is a sparse matrix of high dimension. and as the total number of training samples is limited, it is observed that the random forest model and mlp model is producing the same performance for this task. whereas the embedding generated by the bert model is of 512 dimensions for each meme. the model comparison based on train-set evaluation for the meme classification and cross-validation with @xmath0. as we can see that the overall performance of the tfidf model is slightly better than the sentence transformers. this maybe is due to the difference in the base corpus the bert model is trained on, which is conflicting with the given colloquial tamil corpus. whereas the tfidf and the count vectorization yielded similar results because of the short length of the meme and containing varied usage of words by different users. the performance of the cross validation scores for the different machine learning approaches are tabulated in performance of the test set is given in ## conclusion a meme is a form of media that spreads an idea or emotion across the internet. memes make it even more challenging since they express humour and sarcasm in an implicit way, because of which the meme may not be offensive if we only consider the text or the image. in this paper we proposed a approach for meme classification for tamil language that considers only the text present in the meme. tfidf and n-gram features given slightly better performance than the sentence embeddings. we achieve f1 scores of 0.50 using the proposed approach using the test-set. meme classification performance can be increased by considering visual features also.