BPHC @ D ravidian L ang T ech- ACL 2022-A comparative analysis of classical and pre-trained models for troll meme classification in T amil (2022)

Abstract: Trolling refers to any user behaviour on the internet to intentionally provoke or instigate conflict predominantly in social media. This paper aims to classify troll meme captions in Tamil-English code-mixed form. Embeddings are obtained for raw code-mixed text and the translated and transliterated version of the text and their relative performances are compared. Furthermore, this paper compares the performances of 11 different classification algorithms using Accuracy and F1- Score. We conclude that we were able to achieve a weighted F1 score of 0.74 through MuRIL pretrained model.

## introduction technology is ingrained in every aspect of our lives. we require it to communicate with others and thrive in the modern world. we increasingly rely on textbased mediums to interact with technology every day. hence there is a need for machines to understand natural human languages. however, it is challenging for computers to understand natural languages because of the inherent ambiguity in both the syntax and the semantics of natural language @xcite . with the ease of accessing the internet and the surge in the number of social media platforms, social media has become an essential and influential aspect of everyone's life @xcite . it has also brought about a change in the way regional languages are expressed. native script of the regional language is not used for exchanges on social media platforms @xcite . instead, native speakers use roman script combined with english words or phrases through code-mixing to express their ideas. the text generated by users in social media contains a high amount of spelling mistakes, phonetic typing, wordplay characters, and modern internet slang @xcite @xcite @xcite . this is mainly due to the limitation of the english keyboard and the speed at which the modern world moves. thus, the study of text expressed in code-mixed form is essential @xcite . in this paper, we explain our submission to dravidianlangtech-acl2022 for the task of trollmeme classification in tamil. tamil is a member of the southern branch of the dravidian languages, a group of about 26 languages indigenous to the indian subcontinent @xcite @xcite . the earliest old tamil documents are small inscriptions in adichanallur dating from 905 bc to 696 bc @xcite . tamil, one of the 22 scheduled languages in the indian constitution, was the first to be designated as a classical language of india @xcite @xcite @xcite . we brief on all the embedding techniques like tf-idf, m-bert, muril, indicft, etc., and various classification algorithms, including logistic regression, decision tree, svm, etc., that were implemented in the process. the rest of the paper is organized as follows. section 2 details the related works done in the field, and section 3 describes the dataset used. section 4 expands on the methodology and experimental setup, section 5 discusses the results obtained, and section 6 elucidates the conclusions. ## related work there has been a rapid rise in the number of interesting studies performed in the domain of dravidian code-mixed text analysis in the last few years @xcite @xcite . sub-word level or morpheme level embedding technique obtained using a 1d convolution layer with relu activation was proposed by @xcite . after getting a morpheme-level feature map, a 1-d maximum pooling layer is used to obtain its most prominent features. lstms are used to obtain the connections between each of these features due to their ability to process sequences and retain information. @xcite proposed using tf-idf and m-bert embeddings coupled with classification models like random forest, naive bayes, and multi-layer perceptron for the task of classifying english text code-mixed with dravidian languages as offensive or not-offensive. selective translation and transliteration was performed by @xcite , to convert the whole text to tamil text in native script. xlm-roberta multilingual model was used to obtain embeddings. multiple classification algorithms were used to classify code-mixed text as offensive and not-offensive, and logistic regression was found to perform the best. an ensembling of multiple classification algorithms applied on tf-idf embedding was experimented in @xcite . it was found to perform well for shorter dravidian code-mixed sentences like youtube comments, which can be considered to be similar to meme captions. we hypothesized translation and transliteration should improve the overall classification accuracy of the text when the troll script is code mixed. in addition, using language-specific embeddings would yield an improvement. ## conclusion and future work we compared the weighted f1 score of the various classifiers between the raw text and translatedtransliterated text in tamil-english code mixed troll memes. language-specific word embedding techniques significantly improve the classification metrics like accuracy, macro f1 and weighted f1 scores. with a comparison between the various pre-trained models, muril performs best with an f1 score of 0.74 relative to others. we have only considered text captions and not the images. we hypothesize that a unified framework to combine image with text will yield even better results. in the future, we would want to integrate a deep learning model that could take both the caption text and the images together.