MUCS @ D ravidian L ang T ech2023: M alayalam Fake News Detection Using Machine Learning Approach (2023)

Abstract: Social media is widely used to spread fake news, which affects a larger population. So it is considered as a very important task to detect fake news spread on social media platforms. To address the challenges in the identification of fake news in the Malayalam language, in this paper, we - team MUCS, describe the Machine Learning (ML) models submitted to “Fake News Detection in Dravidian Languages” at DravidianLangTech@RANLP 2023 shared task. Three different models, namely, Multinomial Naive Bayes (MNB), Logistic Regression (LR), and Ensemble model (MNB, LR, and SVM) are trained using Term Frequency - Inverse Document Frequency (TF-IDF) of word unigrams. Among the three models ensemble model performed better with a macro F1-score of 0.83 and placed 3rd rank in the shared task.

## introduction with the overwhelming growth of social media like twitter, facebook, youtube, etc., and the ease with which the information can be shared widely and quickly on these social media platforms @xcite @xcite , creation and sharing of fake news has seen the unprecedented growth. the anonymity of users on social media has given a chance to fake news spreaders to divert people's beliefs, trust, and opinions by intentionally spreading fake information. usually rumors and fake news spread fast and damage personal relationships and social connections @xcite . further, they may also cause anxiety and emotional distress through unfavorable perceptions, scrutiny from the public, and social isolation @xcite . in order to prevent harm and discomfort from fake news, to the users, organizations, and communities, identifying and filtering out such fake news automatically has become the need for the day @xcite . the majority of the fake news detection systems have focused on high-resource languages like spanish and english @xcite giving no or less importance for low-resource dravidian languages, such as tulu, malayalam, tamil, telugu, and kannada, due to lack of resources @xcite . among the low-resource languages, malayalam is relatively spoken by a smaller population in indian states of kerala and the lakshadweep islands @xcite . unlike other dravidian languages, malayalam has its own linguistic complexities, including dialect variations, word semantics, idiomatic expressions, and so on. these complexities can make it harder to process and analyze the malayalam text. as there are no guidelines to create any post/comment on social media, users usually combine words and sub-words belonging to more than one language they know, leading to code-mixed text. further, they may use more than one script to create the post/comment. these factors make it difficult to process the code-mixed texts. learning approaches that work well for monolingual text may not give good results for code-mixed texts. further, there are no pretrained models/specific techniques for code-mixed texts in low-resource languages. the "fake news detection in dravidian languages" shared task organised at dravidian-langtech@ranlp 2023 foot_0 @xcite promotes fake news detection in code-mixed malayalam text. to address the challenges of de-tecting fake news in malayalam, in this shared task, we -team mucs, implemented three distinct models: i) mnb, ii) lr, and iii) ensemble models (lr, mnb, and svm) with hard voting, trained with tf-idf of word unigrams. the rest of the paper is organised as follows: section 2 gives a brief description of the related work. while section 3 explains the methodology, section 4 is about experiments and outcomes. finally, the paper concludes in section 5 with future work. ## related work fake news detection in low-resource languages and code-mixed low-resource languages are getting prominence gradually. researchers have tried to explore several techniques to identify fake news using available benchmarked corpora in low-resource languages. a brief description of few of the relevant works are given below: a novel kurdish fake news corpus created by @xcite to detect fake news in kurdish language consists of two datasets (i) crawled fake news and (ii) texts that are altered from real news. tf-idf of words is used to train ml models (naïve bayes (nb), svm, lr, decision tree (dt), and random forest (rf)) to detect fake news. the svm classifier outperformed all other classifiers with an accuracy of 88.71% for 'crawled fake news' and lr classifier outperformed all the other algorithms on 'texts that are altered from real news' with an accuracy of 83.26%. hegde and shashirekha (2021) explored ensemble (rf, mlp, gradient boosting (gb), and adaptive boosting) model with soft voting for urdu fake news detection. using a combination of tf-idf of word unigrams, character n-grams in the range (2, 3), and fasttext vectors, to train the ensemble model, they obtained a macro f1-score of 0.552 and an accuracy of 0.713%. to detect fake news in malayalam language, bijimol and santhosh ( 2022 features and obtained a macro f1-score of 0.592. in another work, balouchzahi and shashirekha ( 2020 ) developed an ensemble model (mnb, lr, and mlp) with hard voting for the binary classification of fake news in urdu. using term frequency of word and character n-grams in the range (1, 2) and (1, 5) respectively to train the ensemble model, they obtained a macro f1-score of 0.770. the approach to detect fake news in the political domain on the "liar" dataset is proposed by @xcite . they trained ml models (xgb, rf, nb, k-nearest neighbors (k-nn), dt, and svm) using tf-idf of word n-gram features and obtained an accuracy of 75% using xgb and 73% accuracy using both svm and rf models. the related work reveals that several ml algorithms trained with tf-idf of word and char ngrams are explored to detect the fake news in lowresource dravidian languages. however, as the performance of many of the existing works are low, there is scope to develop models to detect fake news in code-mixed low-resource dravidian languages. ## methodology the framework of the proposed ml models to identify fake news detection in code-mixed malayalam text is shown in figure 1 and the steps involved in the proposed methodology are described below: ## experiments and results the goal of the shared task is to classify the given malayalam code-mixed text into "original" or "fake" news. the statistics of the malayalam codemixed dataset for fake news detection provided by the shared task organisers is shown in ## conclusion and future work in this paper, we describe the three models: mnb, lr, and ensemble (lr, rf, and svm) classifiers with hard voting, submitted to the "fake news detection in dravidian languages" at dravidian-langtech@ranlp 2023 shared task. the proposed models are trained with tf-idf of word unigrams, for detecting fake news in code-mixed malayalam texts. among the three models, ensemble model obtained 3 rd rank with a macro f1-score of 0.831. as a future work, fake news detection in low-resource languages like tulu, kannada, and other dravidian languages will be explored.