IIITK @ D ravidian L ang T ech- EACL 2021: Offensive Language Identification and Meme Classification in T amil, M alayalam and K annada (2021)

Abstract: This paper describes the IIITK team’s submissions to the offensive language identification, and troll memes classification shared tasks for Dravidian languages at DravidianLangTech 2021 workshop@EACL 2021. Our best configuration for Tamil troll meme classification achieved 0.55 weighted average F1 score, and for offensive language identification, our system achieved weighted F1 scores of 0.75 for Tamil, 0.95 for Malayalam, and 0.71 for Kannada. Our rank on Tamil troll meme classification is 2, and offensive language identification in Tamil, Malayalam and Kannada are 3, 3 and 4 respectively.

## introduction with the rise of deep learning in the past decade, we have seen some groundbreaking milestones achieved by various machine learning (ml) models being built over the era. the alexnet @xcite architecture achieved the first significant milestone on ilvrsc challenge, which showed the state of the art results at their time of recording on the image classification tasks. although transfer learning was proposed much before in the community, it was not used much due to the lack of computational resources. the success of alexnet reignited the usage of transfer learning @xcite which leverages the knowledge gained by the models having parameters of the size of millions to billions and which are being trained over an enormously large dataset can be further use for fine-tuning it over other computer vision tasks to achieve promising results. another breakthrough in the community was the release of bert @xcite which was being built upon the transformer architecture @xcite with several innovative training strategies at the time of its release topped the glue @xcite benchmarks. the introduction of bert paved the ways for the transfer learning on the natural language processing (nlp) tasks. however, one particular issue with these huge pretrained models was that they were most of the times not available for under-resourced languages. the offensive language identification task aimed to detect the presence of offensive content in texts of such under-resourced languages @xcite @xcite @xcite . the texts were in the native language scripts as well as of code-mixed nature @xcite . code-mixing means the mixing of two or more languages or language varieties in speech @xcite . the meme classification task focused on classifying memes into a troll and not a troll. along with the memes, their extracted captions were also being provided in tamil native scripts and code-mixed nature @xcite . all the three languages kannada, malayalam and tamil for which the datasets were being provided across the two tasks are under resourced in nature and belong to dravidian languages' family @xcite . the dravidian language family is dated to approximately 4500 years old @xcite . in jain and buddhist texts written in pali and prakrit in ancient times, they referred to tamil as damil, and then dramil became dravida and dravidian. tamil language family includes kannada, telugu, tulu and malayalam. the tamil languages scripts are first attested in the 580 bce as tamili foot_1 script inscribed on the pottery of keezhadi, sivagangai and madurai district of tamil nadu, india @xcite . the dravidian languages have its scripts evolved from tamili scripts. in this paper, we showcase our efforts for the two shared tasks on dravidian languages. we participated in the shared task offensive language identification @xcite for tamil, malayalam and kannada. we also participated in the shared task on tamil troll meme classification @xcite task. we have used the transformer and bert based models in our work. this paper summarizes our works on the two shared tasks. each of the following sections is being subdivided further into two sections to detail every aspect of the procedures being worked out on them. ## related work offensive content detection from texts has been part of some conferences as challenging tasks. some of the most popular tasks have been part of the semeval foot_2 organized by international workshop on semantic evaluation. the tasks covered in semeval varied from offensive content detection in monolingual corpus to code-mixed corpus collected from social media comments. however, most of these competitions had their tasks on datasets based on english and other high resourced languages from the west. hasoc-dravidian-codemix-fire2020 foot_3 was one such competition being organized for under-resourced languages such as tamil and malayalam. @xcite at hasoc-dravidian-codemix-fire2020 used traditional machine learning (ml) methods like naive bayes classifier (nbc) , support vector machines (svms) and random forest along with the pretrained transformers models like xlm-roberta (xlmr) @xcite and bert for the offensive content identification in code-mixed datasets ( tamil-english and malayalam-english). arora (2020) at hasoc-dravidian-codemix-fire2020 used ulmfit @xcite to pre-train on a synthetically generated code-mixed dataset and then fine-tuned it to the downstream tasks of text classification. meme classification has been one of the major tasks that various social media platforms aim to solve in the past decade. various methods have been proposed in order to classify memes. however, it is still an unsolved problem because of its multimodal nature, making it extremely hard to have a system correctly classify the memes @xcite . classification of memes needs a visual and linguistic approach to be followed. @xcite did work similar to memes analysis. they used the tumblr dataset using memes, which had images with several tags associated with them. they dealt with texts and visual features independently and built a model to predict sentiments behind those pictures with corresponding tags and obtained good results. using this idea of dealing with different modalities independently, different researchers came forward with different implementations for this problem of memes analysis. this particular problem became a flagship the task of semeval-2020. the team having the best performance for sentiment analysis was obtained by the @xcite where they used several different models for addressing the problem by considering both the modalities and coming up with several combinations using both the modalities. however, the best performance was given by considering the textual features only using bert as the architecture for doing so for sentiment analysis of the memes. the overview paper @xcite of memotion analysis has summarized the works of the participants and gives a clearer picture of the status of the task of memotion analysis its reporting. recently facebook ai conducted hateful memes challenge 5 in 2020 whose winner has achieved the best results using several visual, linguistic transformer models. zhu (2020) used an ensemble of four different visual linguistic models vl-bert @xcite , uniter @xcite , villa @xcite , and ernie-vil @xcite . we have used several pretrained multilingual models in our works and used them to fine-tune the offensive language identification task. most of the work has been done by using huggingface transformers library foot_5 . one key issue with the dataset was the skewness of it. we have used the negative log likelihood (nll) loss with class weights, self adjusting dice @xcite ) (sadice) loss and applied it to address the class imbalance issues with the customised version of models provided by the transformers library. we have also used the original versions of the models available apart from the customised implementations. we have trained each of the models possible combinations as mentioned above and loss functions with each of the language available in the dataset separately. the multilingual models which we have used are multilingual bert-cased (mbert-cased), xlm-roberta and indicbert @xcite . since the indicbert model is comparatively much smaller than mbert-cased and the xlm-roberta models we have not tried to develop a customised version for it and went with the original version. there are two different multilingual bert models available one spans 100 languages, and the uncased version, the other one spans 104 languages and is the cased version. bert's author's recommendations are to use the cased versions, so we have used it in our implementations. for the xlm-roberta model, we have gone with the base model, which covers 100 languages. we have added a fully connected layer of 512 neurons on top of the final transformer layer for the customised versions of these models, which gives us 768-dimensional feature vectors as outputs. this fully connected layer is finally followed by a logsoftmax layer having the number of neurons the same as our output classes. the base layers of the original models in the customised versions of the models were being frozen. the custom models were trained with two different loss functions: one was the negative log-likelihood loss (nll loss) with class weights being applied. the other loss function we have tried is the sadice loss. we have reported our results on the development set in ## conclusion we have presented the iiitk submission to tamil troll meme classification, and offensive language identification shared tasks at dravidianlangtech. we experimented with several different pretrained models for the given datasets with their original and customized architecture and different loss functions. our best model for tamil troll meme classification was multilingual bert-cased, and it gave a weighted average f1 score of 0.55. our best model for offensive language identification was also multilingual bert-cased, and it gave weighted average f1 scores of 0.75 for tamil, 0.95 for malayalam and 0.71 for kannada.