hate-alert@ D ravidian L ang T ech- ACL 2022: Ensembling Multi-Modalities for T amil T roll M eme Classification (2022)

Abstract: Social media platforms often act as breeding grounds for various forms of trolling or malicious content targeting users or communities. One way of trolling users is by creating memes, which in most cases unites an image with a short piece of text embedded on top of it. The situation is more complex for multilingual(e.g., Tamil) memes due to the lack of benchmark datasets and models. We explore several models to detect Troll memes in Tamil based on the shared task, “Troll Meme Classification in DravidianLangTech2022” at ACL-2022. We observe while the text-based model MURIL performs better for Non-troll meme classification, the image-based model VGG16 performs better for Troll-meme classification. Further fusing these two modalities help us achieve stable outcomes in both classes. Our fusion model achieved a 0.561 weighted average F1 score and ranked second in this task.

## introduction over the past few years, social media platforms have been expanding rapidly. users of the platform interact by sharing content to enrich their knowledge and social connections. although most of the content on social media platforms that existed so far was textual, recently, a unique message was born: the meme (chakravarthi, 2020; chakravarthi and muralidaran, 2021). a meme is usually created by an image and a short piece of text on top of it, entrenched as part of the image. memes are generally meant to be harmless and conceived to look humorous, but sometimes, bad actors use memes for threatening and abusing individuals or specific target communities @xcite . such memes are collectively known as offensive/troll memes in social media. trolling is the exercise of publicizing a message via social media that is planned to be abusive, inciting, or threatening to distract, which often has rambling or off-topic content to provoke the audience @xcite . in addition, such memes can be treacherous as they can easily harm the reputation of individuals, famous celebs, political entities, businesses, or social groups, e.g., minorities. although various studies have been conducted to detect offensive posts using different natural language techniques, troll meme classification has not yet been explored. the situation for countries like india is more complicated due to the immense lanuage diverisy foot_0 . the meme in the indian context, can be composed in english, local language (native or foreign script) or in combination of both language and script @xcite @xcite @xcite . this adds another challenge for the troll meme classification. tamil is one of the world's longest-surviving classical languages @xcite . tamil is a member of the southern branch of the dravidian languages, a group of about 26 languages indigenous to the indian subcontinent @xcite @xcite . it is also classed as a member of the tamil language family, which contains the languages of around 35 ethnolinguistic groups, including the irula and yerukula languages @xcite @xcite @xcite . recently, there has been a lot of effort to investigate the malicious side of memes, e.g., focusing on hate @xcite , offensive @xcite , and harmful @xcite memes. however, the majority of the studies are centralized around the english language. further several shared tasks like hasoc 2021 @xcite , dravidianlangtech ## related work this section discusses some of the text-based abusive content detection methods and briefly explains the multi-modal techniques used so far to detect malicious memes. ## dataset description the shared task on troll meme classification in dravidianlangtech2022 @xcite at acl-2022 is based on a classification problem with the aim of moderating and minimizing the offensive/harmful content in social media. the objective of the shared task is to devise method-ologies and vision-language models for troll meme detection in tamil. we show the class distribution of the dataset @xcite in ## methodology in this section, we discuss the different parts of the pipeline that we pursued for the detection of troll meme using the dataset.