Towards Detecting Political Bias in H indi News Articles (2022)

Abstract: Political propaganda in recent times has been amplified by media news portals through biased reporting, creating untruthful narratives on serious issues causing misinformed public opinions with interests of siding and helping a particular political party. This issue proposes a challenging NLP task of detecting political bias in news articles. We propose a transformer-based transfer learning method to fine-tune the pre-trained network on our data for this bias detection. As the required dataset for this particular task was not available, we created our dataset comprising 1388 Hindi news articles and their headlines from various Hindi news media outlets. We marked them on whether they are biased towards, against, or neutral to BJP, a political party, and the current ruling party at the centre in India.

## introduction biased news reporting is a widespread phenomenon present in most of the news circulating today .bias is detected manually, but that is a tedious and timeconsuming task; therefore, automation of bias detection in media articles can prove helpful in verifying these articles for their validity more efficiently. hindi is an indo-aryan language spoken mainly in north india. according to ethnologue list 1 of most spoken languages worldwide, hindi ranks third, and a total of 600.5 million hindi speakers exist in the world 2 . it is also the most spoken language in india with a total of 528,3 million native speakers, which makes up around 43.6 per cent of india's population according to the 2011 census of india 3 . we can observe political bias in news media articles by looking at different factors. we observe biases when the author of the article uses strong language trying to sensationalise an event, is partial to a particular political party, does not give a thorough review of held events etc. the headline in such an article is also essential as it is often filled with bias and is the first thing that catches a reader's attention before they start to read the article. as there is no such dataset annotated for political bias hindi language, we created our dataset by collecting articles and their headlines from different hindi news websites. we then annotated the dataset according to whether the article was biased towards or against bhartiya janta party (bjp, the current ruling party at the centre in india) or was neutral. we present several baseline machine learning and deep learning approaches to detecting political bias on our dataset. we observe that xlm-roberta @xcite ), a transformerbased model, outperforms other baseline models and achieves a score of 83% accuracy, 76.4% f1macro, and 72.1% mcc. the main contributions of our work are as follows: the rest of the paper is organized as follows. we discuss prior work related to bias detection. we describe the proposed dataset and analyse the annotations. we describe our baseline models and compare the performance of our approaches. we discuss the societal impacts of bias detection. we conclude with a direction for future work and highlight our main findings. ## related work detection of bias has been studied before with attempts in detecting media bias and its effects on public perception of news and its impact on sociopolitical events like elections. @xcite developed an lstm network model and used it to detect implicit political bias even in the absence of words that relates to either liberal or conservative ideology on two datasets -the ideological books corpus (ibc) and ontheissues (oti). lim et al. (2020) introduces a news bias dataset with sentence-level bias, which allows the development of approaches of bias detection on articles that have subtle bias. wei (2020) introduces a dataset of 200,00 sentences regarding donald trump and used glove vector embeddings to train cnn and rnn to predict the news source of the sentence. they analyze the top 5-grams with their model to gain meaningful insight into trump's portrayal by different media sources. gangula et al. (2019) created a dataset of news articles and headlines collected from telugu newspapers for bias detection and annotated them for bias towards a particular political party. they also propose a headline attention network model for the detection of bias on their dataset. @xcite worked on detecting subjective bias in wiki neutrality corpus (wnc). they propose bert-based ensemble models for bias detection, which utilizes predictions from multiple models to get better accuracy results. some independent organizations also work to fight misinformation. alt news foot_0 is a fact-checking website that works to debunk misinformation and disinformation on mainstream social media platforms. vishwas news foot_1 is another fact-checking website that is certified by international fact-checking network (ifcn). ## dataset description we have looked at two major types of biases present in an article while annotating: coverage / visibility bias and tonality / statement bias (d'alessio and @xcite . we have annotated our dataset using these biases into 3 categories, biased towards the bjp, biased against the bjp, and neutral if these biases are not visible in the article. ## system description in this section, we describe the data splits we use, the evaluation metrics we consider, and the baselines we propose. ## results and discussion we report the results of our models in to further analyse the results, we compare the class-wise results of the best models. we show the confusion matrix of the predictions compared to the ground truth values in figure 2 . the model is performing very well on the biased classes but suffers heavily on the neutral class. we observe the same pattern during the annotation process and believe that predicting whether an article is unbiased is comparatively more challenging than predicting the type of bias. ## conclusion in this paper, we proposed a dataset to detect biases in hindi news articles. we analysed the difficulty level of each class, and our experiments indicate that detecting whether an article is unbiased is a more challenging problem than detecting the type of bias. further, we provided several baseline models on the proposed dataset and found out that multilingual deep learning models outperform other approaches by a large margin and should be the choice for performance metrics. we perform error analysis on the best performing model to further understand the shortcomings of our proposed system. lastly, we also discussed the ethical and societal implications of the proposed work.