Overview of the Second Shared Task on Fake News Detection in D ravidian Languages: D ravidian L ang T ech@ EACL 2024 (2024)

Abstract: The rise of online social media has revolutionized communication, offering users a convenient way to share information and stay updated on current events. However, this surge in connectivity has also led to the proliferation of misinformation, commonly known as fake news. This misleading content, often disguised as legitimate news, poses a significant challenge as it can distort public perception and erode trust in reliable sources. This shared task consists of two subtasks such as task 1 and task 2. Task 1 aims to classify a given social media text into original or fake. The goal of the FakeDetect-Malayalam task2 is to encourage participants to develop effective models capable of accurately detecting and classifying fake news articles in the Malayalam language into different categories like False, Half True, Mostly False, Partly False, and Mostly True. For this shared task, 33 participants submitted their results.

## introduction the second shared task on fake news detection in dravidian languages, held at dravidian-langtech@eacl 2024 1 , is a significant initiative in the field of natural language processing (nlp) and computational linguistics. this research article provides an overview of this event, which focuses on developing and evaluating techniques for identifying fake news specifically in the dravidian language family. the task aims to address the growing challenge of misinformation in online content by leveraging the linguistic characteristics unique to dravidian languages. by highlighting the importance of language-specific approaches to fake news 1 https://codalab.lisn.upsaclay.fr/ competitions/16055 detection, this work contributes to advancing the capabilities of nlp models in combating misinformation across diverse linguistic contexts. fake news detection (fnd) can be categorized as either monolingual or multilingual. monolingual fnd focuses on detecting fake news in a single language, while multilingual fnd involves identifying deceptive content that may involve code-mixing or code-switching across two or more languages. detecting fake news in low-resource languages poses challenges due to limited language resources such as annotated datasets, language models, and pretrained embeddings. nevertheless, there are strategies to enhance fake news detection in these languages, including the collection and annotation of data, utilization of cross-lingual models, implementation of transfer learning, and the development of domain-specific models. in the study conducted by @xcite , they employed two pre-trained transformer models namely mbert and xlm-r to assess the feasibility of transfer learning from high-resource languages to low-resource languages in the context of fake news detection within dravidian languages. a multichannel cnn can be used to teach the model a variety of features from multiple perspectives. according to @xcite , the model's several channels each extract features from the same input in a different way, resulting in a more accurate representation. a two-stage hope detection approach was proposed by @xcite . in this approach, the language detector determines the model's language, and the hope detector categorizes the text as either hope speech, non-hope speech, or not lang. the authors ahmad et al. (2020) used textual features in conjunction with machine learning-based ensemble algorithms to differentiate between authentic and fraudulent news. ## task description task 1: the goal of this task is to classify a given social media text into original or fake. the data sources are various social media platforms such as twitter, facebook, etc. given a social media post, the objective of the shared task is to classify it into either fake or original news. for example, the following two posts belong to fake and original categories, respectively. this is a comment/post-level classification task. given a youtube comment, the systems submitted by the participants should classify it into original or fake news. to download the data and participate, go to the participate tab. task 2: the fake news detection from malayalam news (fakedetect-malayalam) shared task provides a platform for researchers to address the pressing challenge of identifying and flagging fake news within the realm of malayalam-language news articles. in an age of information overload, accurate detection of misinformation is crucial for fostering trustworthy communication. the core objective of the fakedetect-malayalam shared task is to encourage participants to develop effective models capable of accurately detecting and classifying fake news articles in the malayalam language into different categories. here, we considered five fake categories -false, half true, mostly false, partly false, and mostly true. ## dataset the dataset described in the provided out of all the models considered, the team specifically focused on a fine-tuned muril bert model for the submission and demonstrated an f1 score of 0.86. ## results the rank list for task a and task b with macro f1-score is shown in from the submissions, it is interpreted that transformer-based models outperform machine learning and deep learning models with better f1-score. a few methods based on rnn and cnn gave slightly lesser f1-score than transformer models. following the rnns and cnns, the machine learning models such as svm, naive bayes, random forest, and logistic regression gave f1-score ranging from 0.80 to 0.71 ## conclusion this paper presents an overview of the second shared task on fake news detection in dravidian languages -dravidianlangtech@eacl 2024. the task attracted participation from eighteen teams for task a and twelve teams for task b. they employed methods varied among the teams, ranging from traditional tf-idf vectorizers with machine learning to contemporary pre-trained transformer models for data representation. upon analyzing the methodologies, a consistent trend emerged: transformer-based approaches consistently outperformed other techniques, as evidenced by evaluation metrics like classification accuracy and confusion matrices. this underscores the effectiveness of transformer models in capturing the performance of fake news detection. to sum up, the paper summarizes the dravidianlangtech@eacl 2024 fake news detection shared task for malayalam, emphasizing diverse strategies and highlighting the prevalence of transformer-based methods for enhanced performance.