Augmented Political Leaning Detection: Leveraging Parliamentary Speeches for Classifying News Articles (2024)

Abstract: In an era where political discourse infiltrates online platforms and news media, identifying opinion is increasingly critical, especially in news articles, where objectivity is expected. Readers frequently encounter authors’ inherent political viewpoints, challenging them to discern facts from opinions. Classifying text on a spectrum from left to right is a key task for uncovering these viewpoints. Previous approaches rely on outdated datasets to classify current articles, neglecting that political opinions on certain subjects change over time. This paper explores a novel methodology for detecting political leaning in news articles by augmenting them with political speeches specific to the topic and publication time. We evaluated the impact of the augmentation using BERT and Mistral models. The results show that the BERT model’s F1 score improved from a baseline of 0.82 to 0.85, while the Mistral model’s F1 score increased from 0.30 to 0.31.

## introduction in an era increasingly dominated by digital landscapes, political discussion has largely migrated online. as readers engage with posts and news articles, they encounter not only the facts but also the authors' inherent viewpoints. this phenomenon is further complicated by the prevalence of misinformation and disinformation, making it difficult for individuals to maintain objectivity and discern the embedded viewpoints. the presence of political viewpoints in news articles poses significant challenges for the average reader, potentially leading to cognitive biases. according to a recent study by @xcite , readers, among others, fall for confirmation bias. it reinforces pre-existing beliefs, as users tend to agree with statements that foster their own opinions. detecting political viewpoints in textual content is thus crucial for ensuring that readers can access a comprehensive view of the information presented. political opinion can be categorized differently. a common way is to categorize text on a left-toright spectrum, known amongst others as political leaning detection @xcite . previous research on political leaning has predominantly focused on english-speaking contexts, yet such models are not directly transferable to the german linguistic and political landscape, where left and right viewpoints on specific topics vary significantly. for example, topics considered rightleaning in the united states, such as gun ownership rights, may not be framed similarly in germany @xcite . moreover, the dynamic nature of political landscapes means that older datasets might not accurately reflect current political climates or the evolving positions of political parties. the recent shifts in german politics with the emergence of the party alternative für deutschland (afd) in the german federal parliament (bundestag) illustrates these changes, necessitating updated and relevant datasets for analysis. furthermore, there are differences in legislation terms as the opposition parties usually use more emotional language in their speeches when criticizing the government parties @xcite . this paper proposes a novel system for detecting political leaning. it leverages current speeches from the bundestag to improve the classification of ger-man news articles by augmenting the articles with timely quotes from right-leaning and left-leaning speeches. ## related work a political viewpoint is defined as one of a limited number of identifiable opinions on a political subject. in the realm of automatic political viewpoint detection in english text, terms like ideology, leaning, party, and political bias are frequently used interchangeably, distinguishing classes typically along a left-to-right spectrum @xcite . although the specific characteristics of left and right may vary in german text, research indicates a consistent association across countries. the left is typically linked to egalitarianism, progress, social freedom, internationalism, and state intervention in the economy, while the right is associated with tradition, authority, nationalism, and market liberalization @xcite . for clarity, we will use the term political leaning to denote a viewpoint positioned on the spectrum between left and right ideologies. numerous studies have employed traditional machine learning models to classify political leaning @xcite @xcite @xcite . due to the recent advancements in deep learning, the utilization of large language models (llms) is increasingly coming into focus. roberta @xcite in combination with a back-translation technique for augmentation proved to be effective in the clef checkthat! shared task on political bias detection @xcite . @xcite proposed another augmentation method where text samples are expanded by samples with the same target and bias. varies research was conducted based on the idea of comparing left and right viewpoints on the same topic to highlight differences. in 2020, roy and goldwasser (2020) examined the appearance of words from politically opposed news outlets regarding specific topics. later, @xcite improved classification by pre-training an llm with article pairs that presented the same story but from different ideological perspectives. a recurring challenge in the classification of news articles is the inherent media bias. hong et al. (2023) proposed using multi-head attention mechanisms, applying both document-level and sentence-level labels to ensure that the sentences accurately reflect the document's overall political leaning. moreover, techniques such as adversarial media adaptation and triplet loss pre-training are introduced by @xcite to mitigate the influence of media bias on model outcomes. @xcite suggested modelling article-level bias by analyzing sentence-level bias along with other features like frequency, position, and sequence of biased terms. incorporating political speeches has shown promising results in enhancing the performance of political leaning detection models. @xcite analyzed the cosine similarities between articles and speeches to classify news outlets along a political spectrum, confirming the relevance of political speeches. @xcite applied logistic regression models trained on bag-of-words vectors from german political speeches. @xcite aligned social media posts with congressional speeches using tf-idf similarity. in this study, we augment a news article dataset with text from political speeches. building on the ideas of @xcite and @xcite we leverage discussions on the same topic from different parties in the bundestag. by using sequence pairs from the furthest left and furthest right parties in the parliament, we train the model to recognize the two most divergent political viewpoints on a given topic. ## datasets we use two corpora, a news article dataset from @xcite and german parliament speeches from open discourse @xcite to evaluate our proposed approach. ## models we utilize a small encoder model and a small decoder model to demonstrate the effectiveness of speech augmentation. for the encoder, we fine-tune a bert model @xcite pre-trained on a german corpus. to incorporate larger generative models and the concept of retrieval augmented generation, we conduct inference tests with an llm equipped with a decoder. we use mistral-7b-instruct @xcite as it is a small european llm and is fine-tuned on instructions, which helps to retrieve valid json outputs. the prompt details for mistral are provided in the appendix. both models are designed as classifiers. bert is modified by appending a dropout and a linear layer, while mistral utilizes few-shot learning to output a json containing the label prediction. we then compare the baseline models of bert and mistral against the augmented versions. for bert, we enhance its capabilities by expanding it with a cross-encoder model, allowing it to incorporate the augmentation sequences. comparing left-leaning and right-leaning sequences may help to interpret the opinions expressed in the news article. cross-encoders are typically used for similarity calculation in retrieval processes. we exploit this feature to encode contrasting sequences. it takes two sequences as inputs, and we take its last hidden states as input for the dropout layer. the bert model baseline and expanded bert model are illustrated in figure 1 . in the case of the mistral model, the augmentation process is straightforward. left and rightleaning sequences are included in the instructions, as detailed in the appendix 5. ## experimental setup we use the german bert base model foot_0 and a crossencoder for multilingual support foot_1 . for mistral, we use mistral-7b-instruct foot_2 . due to input size limitations in both models, we use a maximum token size of 512 for the articles. for evaluation, we train our bert models with a train-eval-test split of 70%, 15% and 15%. for mistral, we perform few-shot learning by adding a left-leaning and right-leaning news article to the prompt and running it on the test set only. for evaluation metrics, we use marco precision, recall and f1 score. we fine-tune the bert models using grid search and run 20 random parameter variations. we always train 4 epochs and use batch sizes @xcite @xcite , learning rates [1e-5, 2e-5, 3e-5, 4e-5, 5e-5], weight decays [0.01, 0.1], warm-up ratios [0.06, 0.08, 0.1] and dropout rates of [0.1, 0.2, 0.3, 0.4]. ## results in contrast, the five-class classification shows varied performance, with the 'far-left' class failing to identify any true positives (precision and recall at 0.00), while 'center-left' and 'center-right' classes have strong performances with f1-scores of 0.99 and 0.94, respectively. the 'center' and 'far-right' classes show moderate results, contributing to an overall average f1-score of 0.68 for the model. ## discussion and limitations the results from mistral appear nearly random, indicating significant room for improvement. currently, it operates primarily through inference, which might be inadequate. fine-tuning mistral on a specific task using the training dataset could enhance its performance. data augmentation didn't significantly improve mistral's performance, potentially because mistral was trained on bigger and more current datasets than models like bert. the choice of using a cross-encoder to enhance an encoder model warrants evaluation. testing different configurations could reveal more effective alternatives. for instance, employing an encoder with a larger input size and directly appending speech augmentation to article tokens might optimize performance. our approach may reduce media bias by utilizing non-media datasets for comparison. however, the challenge of mitigating media bias persists due to our reliance on datasets labelled based on news outlets. experimenting with a manually labelled german news dataset could provide a clearer indication of the effectiveness of our methodology. the process of augmentation and similarity search has limitations. to improve, methods such as isolating subjective sentences could be investigated. furthermore, considering speeches from a narrower time frame, such as within a specific twomonth period, might yield more precise insights. our analysis currently limits itself to speeches from linke and afd. this restriction might oversimplify the complex political spectrum. incorporating speeches from a broader range of political parties would add complexity and could provide a more comprehensive understanding. however, categorizing parties on a simple left-right scale is challenging as multiple dimensions influence political parties. shared viewpoints among opposing parties might undermine the utility of the left-right scale. it raises the question of whether detecting political leaning or ideology is more suitable for analyzing the german political landscape. ## conclusion the paper represents an initial attempt to use speeches to augment articles for the detection of political leaning. we explore this approach with two classic llm methods: fine-tuning bert and fewshot learning with mistral. the study incorporated sequences from the most left-leaning and most right-leaning parties as a thematically and temporally relevant input for the articles. for both models, there were slight improvements in f1 scores, with bert increasing from 0.82 to 0.85 and mistral from 0.30 to 0.31. further research is necessary to determine the effectiveness of augmentation.