Tayyab@ D ravidian L ang T ech 2024:Detecting Fake News in M alayalam LSTM Approach and Challenges (2024)

Abstract: Global communication has been made easier by the emergence of online social media, but it has also made it easier for “fake news,” or information that is misleading or false, to spread. Since this phenomenon presents a significant challenge, reliable detection techniques are required to discern between authentic and fraudulent content. The primary goal of this study is to identify fake news on social media platforms and in Malayalam-language articles by using LSTM (Long Short-Term Memory) model. This research explores this approach in tackling the DravidianLangTech@EACL 2024 tasks. Using LSTM networks to differentiate between real and fake content at the comment or post level, Task 1 focuses on classifying social media text. To precisely classify the authenticity of the content, LSTM models are employed, drawing on a variety of sources such as comments on YouTube. Task 2 is dubbed the FakeDetect-Malayalam challenge, wherein Malayalam-language articles with fake news are identified and categorized using LSTM models. In order to successfully navigate the challenges of identifying false information in regional languages, we use lstm model. This algoritms seek to accurately categorize the multiple classes written in Malayalam. In Task 1, the results are encouraging. LSTM models distinguish between orignal and fake social media content with an impressive macro F1 score of 0.78 when testing. The LSTM model’s macro F1 score of 0.2393 indicates that Task 2 offers a more complex landscape. This emphasizes the persistent difficulties in LSTM-based fake news detection across various linguistic contexts and the difficulty of correctly classifying fake news within the context of the Malayalam language.

## introduction online social media has made communication easier on a global level, which allows people to seam-1 https://codalab.lisn.upsaclay.fr/competitions/16055 lessly interact and share information with each other across the globe. in the realm of natural language processing (nlp) @xcite , diverse tasks hold significance, ranging from detecting hate speech @xcite and hopeful @xcite sentiments @xcite to language identification @xcite and combatting fake news. researchers leverage various models tailored to these tasks' intricacies. for hate speech detection @xcite , models like convolutional neural networks (cnns), recurrent neural networks (rnns), traditional machine learning @xcite , and transformerbased @xcite architectures such as bert and its variants have been instrumental. on the other hand, the spread of false information and fake news @xcite is a serious problem brought about by this increased connectivity. significant doubts about the veracity and credibility of online content have been raised by the purposeful spread of inaccurate or misleading information through a variety of social media platforms. this trend has major ramifications for society cohesion, public confidence, and democratic discourse in addition to endangering the veracity of information. the dravidianlangtech@eacl 2024 @xcite task was created for this problem, with a specific focus on countering fake news in dravidian languages. the goal of this innovative project is to use advanced technology, specifically lstm (long short-term memory) models @xcite , to address the complex problems related to identifying and categorizing fake news @xcite . this initiative aims to achieve two main goals. initially, task 1 focuses on social media content classification, with a focus on distinguishing between accurate and false information. the challenge for participants is to create lstm-based systems that can operate at the comment or post level on social media sites like facebook, youtube, and twitter and can distinguish between authentic content and fake content. second, the goal of task 2, also referred to as the fakedetect-malayalam challenge is to recognize and classify fake news in articles written in malayalam. the task for participants is to create lstmbased models that can effectively identify false information in malayalam and categorize articles into groups such as mostly true, false, half true, mostly false, and partly false. the initiative aims to progress fake news detection, particularly in the context of dravidian languages, through these tasks. the results and developments that come from this work will not only help create strong detection systems but also promote credibility, trustworthiness, and dependability in online content, forming a more genuine and informed digital environment. ## related work trends in technology and extensive research have been made in the field of fake news detection in response to the spread of misinformation and fake news on online platforms in recent years. various approaches, strategies, and methods have been investigated in a number of studies to address the widespread problem of fake news, which includes social media platforms and multilingual environments. one popular area of research has been identifying fake news on social media. by using neural networks to detect fake news on twitter, @xcite invented the application of deep learning @xcite techniques @xcite .they showed how machine learning models can effectively separate false information from real content by focusing on feature extraction and classification in their study. furthermore, @xcite suggested a method for detecting fake news by analyzing the text of social media posts using natural language processing (nlp) techniques @xcite . their research highlighted the value of sentiment analysis and linguistic characteristics in precisely detecting false information. moreover, studies have begun to focus on multilingual settings, recognizing the difficulties presented by false information in tongues other than english. @xcite examined the transferability of models across languages in their investigation of cross-lingual fake news detection. @xcite research on the identification of fake news in dravidian languages is beginning to emerge. in their 2020 study, vigneshwaran and soman investigated the detection of fake news in tamillanguage news articles by using machine learning algorithms to categorize the authenticity of the @xcite . their research highlighted how crucial tamil-specific linguistic subtleties are to creating precise detection models. furthermore, the use of lstm model has become more popular in the identification of fake news. recurrent neural networks are effective at capturing temporal dependencies and contextual information in text; @xcite used lstm networks to detect fake news in chinese social media. @xcite ) furthermore, the focus has been directed to initiatives aimed at addressing the detection of fake news in languages like malayalam. kunnath and jayaraman @xcite used machine learning models in conjunction with lexical and syntactic features to study the detection of fake news in malayalam @xcite . their research demonstrated the importance of using language-specific strategies to effectively counteract misinformation. in support of this research work, the dravidian-langtech@eacl 2024 initiative seeks to expand the reach of fake news detection to dravidian languages. this initiative addresses a research gap by integrating the lstm model concentrating on social media content and malayalam-language articles. overall, these studies present an overview of the dravidianlangtech@eacl 2024 initiative's pursuit of combating fake news within dravidian languages by highlighting the various methodologies and approaches used in fake news detection across social media platform. ## task description this work aims to identify fake news detection in dravidian languages, as mentioned in the introduction . there are two sub-tasks in this work. ## methodology due to the complex nature of data for both tasks, it is quite obvious that the proposed model must have different aspects to precisely and accurately predict the fake and original content and similarly for multi-class classification for the second task. ## results and discussions different performance outcomes were found when our lstm-based models were evaluated for both tasks. with an macro f1-score of 0.78, our model shown encouraging performance in task 1, which focused on differentiating between real and fake news (binary classification). due to the presence of dropout layers and proper pre processing, the model was able to identify unique patterns within textual sequences, which resulted in a balanced performance that showed notable results. on the other hand, our lstm model faced a more difficult environment in task 2, which faced multi class classification. with an f1-score of 0.2393, the model encountered difficulties in accurately categorizing news articles into multiple classes. ## error analysis the lstm model for malayalam fake news detection demonstrates remarkable accuracy, particularly in true positive identification. however, a notable challenge arises with false positives, mislabeling instances as fake news even in a balanced dataset. this pattern warrants meticulous analysis and adjustment in the model's discriminatory capabilities. comprehensive evaluations on validation and test sets are crucial for assessing the model's adaptability. proposed strategic modifications involve fine-tuning parameters and scrutinizing false positive occurrences to enhance overall accuracy and efficacy. ## limitations utilizing the model lstm in fake news detection offers improved textual comprehension, but effectiveness may be limited by corpus specificity. finetuning is crucial to address potential mismatches with unique malayalam fake news characteristics. the linguistic complexities of malayalam may hinder the model's ability to discern subtle patterns, requiring further investigation and refinement. ## conclusion in conclusion, task 1 and task 2 evaluation of our lstm-based model highlights both achievements and weaknesses. task 1 showed excellent performance, obtaining a noteworthy macro f1-score of 0.78 in binary classification, successfully differentiating real news from fake news. this success confirms the lstm model's ability to identify distinct patterns in textual sequences and shows its ability to accurately classify binary data. with an f1-score of 0.2393 task 2, which involved multi-class classification, highlighted weaknesses. the model had difficulties correctly classifying content multiple classifications, indicating that it was not able to differentiate between different categories. this emphasizes that in order to increase multi-class classification skills, feature representation, data balance, or model refinement changes are required. the differences in the tasks show the effectiveness of lstms in binary classification as well as the challenges that arise in multi class classification. resolving these complexities requires concerted efforts, such as advanced feature engineering, possible data balancing techniques, or model improvements targeted at improving multi class classification. as we proceed, an iterative process that includes extensive testing, enhanced feature representations, and model optimizations is important.