TRO ( F ) LL or ROFL ? : Exploring Troll Detection in T amil Memes (2024)

Abstract: The advent of social networks has deeply improved and enhanced the ways in which people communicate. However, along with the positives, there are negatives as well. The rapid dissemination of information via various means, be it tweets, Whatsapp forwards or memes has led to widespread misinformation and online abuse. The increasing prevalence of misinformation and

## introduction social media has revolutionized the way that people interact with and share content. it offers a platform for people to express their ideas and opinions freely without strict surveillance as is the case in media sources like television, radios and newspapers. while freedom of expression is vital, leaving it unchecked can lead to people taking advantage of it. although most posts on the internet are lighthearted and harmless, with most being targeted at current events to get more eyes on them, sometimes people cross the fine line between humour and insensitivity and hurt others. as social media continues to evolve, fostering a respectful environment while preserving open dialogue remains a crucial challenge for both platforms and users alike. trolling is a form of online bullying that involves harassing, criticizing, or antagonizing someone through provocatively disparaging or mocking public statements, postings, or acts. the one performing these acts is known as a troll @xcite . identifying whether or not a meme is a troll or not is still not as accurate as it can be @xcite despite recent advancements in the field of natural language processing. @xcite studied the variation of memes, and according to them a meme combines images with witty phrases and/or sarcastic or humorous text, making it clear that this is an image classification problem. memes have come to be the most passiveaggressive way to threaten or harm people nowadays. in a country as diverse as india, with multiple languages being spoken in each state, sharing memes in low resource languages has come to be a growing concern. exploiting the lack of familiarity with certain languages, these memes, disguised as humorous content, can spread harmful or offensive messages without detection. this not only complicates moderation efforts but could also deepen social divides, as harmful content flies under the radar in regional dialects, evading scrutiny while still inflicting damage on specific communities. memes are not always written in regional languages. they could also be transliterated or feature a mix of both english as well as another language. figure 1 : a scene from "manadhai thirudivittai" figure 1 is an image of the tamil comedian vadivelu with a smug expressian and the text in tamil translates to "the one who doesn't study is putting his fingerprint on the paper (because he does not know to write)... the one who studies is putting his fingerprint on the phone (modern security mechanisms)... that's the whole point." this suggests a comparison between two types of people: those who make an effort (the one who studies) and those who do not (the one who doesn't study). it highlights a humorous or sarcastic observation about how some people might claim credit or success without actually doing the work, while others who put in the effort are recognized in a different way. figure 2 is a scene from the hugely popular tamil movie "friends". the image is from a scene featuring tamil actors surya(in white) and comedian vadivelu(in blue). in the image, surya who is scrubbing the wall is working slowly. vadivelu asks him "why are you rushing, slow down you have one and a half years until diwali. keep it slow and steady. such a mess, such a mess". the meme is a sarcastic jab at him for working so slowly that there is no need to worry about deadlines, that has been exaggerated for comedic effect. this is an example of a meme that is not harmful, and is meant to be seen and laughed at without offending or mocking anyone. memes are ubiquitous on the internet, but there is no satisfactory way to classify the memes as "troll" or "not troll". this work uses the dataset created by @xcite on two mod-els, and compares their accuracy. the findings have been summarized using precision, recall, and f-score metrics. ## troll meme a troll meme is a form of internet content that combines humor, sarcasm, or provocative elements with the intent to offend, provoke, or elicit strong reactions from a target audience. unlike traditional memes or non-troll memes that aim to entertain, troll memes are created to manipulate emotions, often stepping into offensive territory. they might use images and text in various combinations such as offensive text paired with neutral images, or benign text coupled with disturbing visuals to create a jarring contrast. the underlying goal of troll memes is to distract or disrupt conversations, provoke anger, or mock individuals, groups or social issues. these memes often thrive on the anonymity and rapid sharing that social media platforms provide, making them a powerful tool for spreading disruptive content. figure 3 illustrates an example of a trolling meme targeting the tamil television channel 'vijay tv' for repeatedly airing the same movies. the translation of the text mocks the way the channel celebrates bogi festival. the exact translation says "discharge the old patients raja rani, bahubali, saatai which refer to the names of the movies that have been aired by the channel multiple times in the past. admit the new patients pariyerum perumal, vadachennai, saamy 2 referring to the new movies that are going to be aired. the template is from the movie vasool raja mbbs which is used to mock the channel for its repeated airing tendency. this meme is classified as a troll meme as it tries to tamper the reputation of the television channel. similarly, figure 4 presents a trolling meme directed at the character 'vijay' from the movie theri. the reference image captures a scene where vijay confronts the antagonist. the translated text reads: 'he (the antagonist) didn't even know you were the one who killed his son. so why did you reveal it yourself and invite trouble, ultimately leading to samantha's (the female lead) death?'" in conclusion, both examples demonstrate how troll memes leverage humor, sarcasm and cultural references to criticize or provoke, often targeting popular media figures or institutions. these memes are a powerful tool for social commentary but can also serve to damage reputations or stir controversy. figure 4: example of troll meme 2 3 literature survey recent research on online trolling and aggression has highlighted various aspects of this complex issue. (suryawanshi et al., 2020 2020 ) created a corpus for sentiment analysis in code-mixed tamil-english text, while @xcite looked into tamil-english cross-lingual information retrieval. @xcite used three different transformer models namely bert, albert and xlnet on the same dataset to try and classify memes as troll or not troll. finally, @xcite highlighted the importance of generating bilingual texts for cross-lingual fertilization. this body of work underscores the necessity for improved tools and datasets to effectively address trolling and aggression in diverse linguistic environments. ## dataset the dataset consists of a variety of images containing memes in tamil, transliterated tamil, as well as english, collected by @xcite . these memes include sentences in monolingual tamil, transliterated tamil in roman script, code-mixed (combining tamil and english within a single sentence), and code-switched (alternating between tamil and english at phrase or sentence boundaries) as shown in in the figure 3 . there are 2,289 memes in the training set, with 1,251 classified as troll images and 1,038 classified as not troll images. the test dataset comprises 659 images, with 389 being troll and 270 being not troll. this dataset is slightly imbalanced, with troll images being more frequent (about 55% vs. 45%) than non-troll images. the presence of code-mixed and code-switched text further adds to the complexity of the dataset, posing unique challenges for preprocessing and classification tasks.f 5 model architecture this paper makes use of the muril model that is developed specifically for processing code-mixed and colloquial language like tamil with codeswitching. it builds upon the bert architecture but adds modifications that can better capture the linguistic nuances that are seen in indian languages, particularly in informal, mixed-language contexts. this makes muril especially well-suited for detecting complex phenomena such as codeswitching (i.e., the alternation between languages within a sentence) and code-mixing (i.e., the intermingling of words from multiple languages). the muril model is configured with key hyperparameters that enhance its ability to handle the nuances of colloquial language. this means that an embedding size of 768, accompanied by a vocab size of 197,285, would allow the model to efficiently capture words and contextual meanings in the source as well as target languages. with an intermediate size of 3072 and 12 attention heads in self-attention mechanism, this would allow the model to process relationships between words efficiently and capture context and meaning for mixedlanguage phrases. the use of 12 layers in the encoder as well as max position embeddings up to 512 ensures a longer sequence would not reduce important sequential information in its processing. furthermore, various regularization techniques in the forms of hidden dropout probability, attention dropout that are 0.1 help keep the model free from the problem of overfitting, thus providing generalizations even when considering noisy or informal language usual in data in real scenarios. these hyperparameters are therefore critical to muril's ability to be able to identify linguistic patterns and anomalies in code-mixed tamil sentences, particularly for the task at hand such as abusive language or hate speech, since these often use a blend of languages. this sets up muril appropriately to deal with the complexity involved in colloquialism in code-switching or code-mixing cases, hence it is aptly suited for this kind of model. ## methodology this work used @xcite to train the datasets. muril (multilingual representations for indian languages) is a pre-trained language hyperparameter value _name_or_path hate-speech-cnerg/tamil-codemixed-abusive-muril embedding_size 768 hidden_size 768 num_attention_heads 12 num_hidden_layers 12 intermediate_size 3072 max_position_embeddings 512 vocab_size 197285 attention_probs_dropout_prob 0.1 hidden_dropout_prob 0.1 initializer_range 0.02 torch_dtype float32 transformers_version 4.46.3 the difference between both approaches is that the first model used train_test_split where the dataset is split into 80% training and 20% validation. the model is trained and evaluated only once on this specific split. it does not use early stopping, meaning the model always trains for the full number of epochs. the second model used k-fold cross validation with early stopping where the dataset is split into 5 subsets (folds), and the model is trained 5 times, each time using 4 folds for training and 1 fold for validation. this approach evaluates the model across multiple splits of the dataset, leading to a more reliable estimate of model performance. training stops early if validation loss does not improve for 1 epoch, potentially saving time and preventing overfitting. the model is evaluated on each fold's validation set after training, and the results are recorded for each fold. ## results and discussion the train-test split model, as shown in for the "troll" class, the model exhibited poor performance, with a recall of only 0.11, meaning it correctly identified only 11% of actual troll memes. although the precision was 0.50, the low recall and f1-score of 0.17 highlight the model's struggle in accurately detecting troll memes. this imbalance indicates a significant limitation in its ability to effectively classify troll content, which is crucial for the task at hand. on the other hand, the k-fold cross-validation model with muril, shown in for the "not troll" class, the model's performance was weak, with a precision of 0.52 and an extremely low recall of 0.04, meaning it identified only 4% of actual non-troll memes. the f1-score of 0.08 underscores this poor balance between precision and recall, as 258 of the 270 "not troll" instances were misclassified as "troll." in contrast, the "troll" class showed much stronger performance, with a recall of 0.97 indicating that 97% of troll memes were correctly classified. the precision of 0.59 suggests that while the majority of troll predictions were accurate, some false positives still occurred. overall, the f1-score of 0.74 reflects good performance for this class. from the model's performance clearly exhibits an imbalance, effectively identifying troll memes but struggling significantly to recognize non-troll content. this discrepancy between the two classes suggests the need for further improvements, particularly in enhancing the model's ability to distinguish non-troll memes.