CL o SE : Contrastive Learning of Subframe Embeddings for Political Bias Classification of News Media (2022)

Abstract: Framing is a political strategy in which journalists and politicians emphasize certain aspects of a societal issue in order to influence and sway public opinion. Frameworks for detecting framing in news articles or social media posts are critical in understanding the spread of biased information in our society. In this paper, we propose CLoSE, a multi-task BERT-based model which uses contrastive learning to embed indicators of frames from news articles in order to predict political bias. We evaluate the performance of our proposed model on subframes and political bias classification tasks. We also demonstrate the model’s classification accuracy on zero-shot and few-shot learning tasks, providing a promising avenue for framing detection in unlabeled data.

## introduction news media coverage shapes our attitudes, emotions, and decisions toward public issues @xcite @xcite . research shows that people's perceptions of news can be manipulated by presenting the same story with different expressions. for example, participants of a study found a terrorist attack caused by "al-qaeda and associated radical islamic groups" considerably more threatening than a terrorist attack by "domestic rebel separatist groups," which is an equivalent paraphrase @xcite . hence, studies on the influences of different presentations of issues, or the effects of framing, play an essential role in understanding political discourse. framing refers to emphasizing desired aspects of an issue to promote and amplify a particular perspective @xcite . by selecting and thus elevating the salience of a specific angle of a topic, media sources can present the topic through their choice of frames to induce particular attributes and judgments among the public. framing is widely re-searched on various topics, from its effects on public opinion on political issues such as the u.s. antinuclear war movement @xcite and stem cell research @xcite to the economic impact of framing @xcite . in this work, we study framing detection for three politically polarized issues in the u.s. news media: abortion, gun control, and immigration. we focus on framing in the news discourse to understand a discrepancy in media consumption and its influence on political bias polarization. with the increase of news media outlets and social media platforms, the public is overwhelmed with a flood of information. unfortunately, the increase of news sources does not yield the sharing of information across political views, often developing into biased echo chambers on social media platforms. in fact, there are stark differences in the social media and news sources that liberals and conservatives use and trust. fox news is the primary news source for nearly half of conservatives, while npr, msnbc, and the new york times are the most trusted news sources for liberals @xcite . machine learning techniques have been applied to detect and analyze political frames @xcite @xcite . in such framing analyses, the performance of a framing detection model is tested by predicting the political bias of an article or the political affiliation or stance of a politician's tweet or speech. however, we suggest incorporating the political bias information into the development of the actual frame detection model. we focus on general liberal and conservative bias rather than the specific stances politicians may take on issues. in this paper, we propose close, a framing embedding model that jointly learns to predict political bias. close fine-tunes bert variants with a contrastive learning objective to generate (sub)frame embeddings for a given input sentence. then we add a prediction loss to classify the political bias (liberal or conservative) of the embedded text. for the embedding task, we use contrastive learning to place embeddings of the same subframe closer together. subframes are fine-grained subcategorizations of the general political frames of @xcite . the topic-specific lexicons of subframes are the subframe indicators used to identify specific framing language within a chosen topic. we use the subframe indicators to identify texts with frames and construct the framing triplet dataset. this dataset, built explicitly for a contrastive objective, consists of a triplet of an anchor sentence, its positive sample, and its negative sample, and the political bias label of the anchor sentence. the contrastive learning objective reduces the distance between the anchor sentence and its positive sample, which belongs to the same subframe, while increasing the distance to its negative sample that belongs to a different subframe. close outperforms the baseline models in both the subframe and political bias classification tasks. the results also show that the contexts of subframes improve the performance of the political bias classifier. further, our pre-trained model accomplishes superior performance in zero-shot and few-shot settings. finally, we design a topic modeling method for the subframe embeddings that clusters nearby embeddings, extracts statistically significant words with class-based tf-idf @xcite , and merges clusters with high overlapping words. to summarize, our main contributions are as follows: (1) we collect and release the triplet framing dataset 1 , a triplet of sentences that include subframe indicators. (2) we propose close, contrastive learning of subframe embeddings model that jointly generates embeddings and predicts political bias of framed texts. the experiments demonstrate our method's performance on subframe and political bias classification tasks and investigate the effectiveness of political bias information on subframe detection and vice versa. (3) the experiments show that our pre-trained model performs competitively on zero-shot political bias classification and few-shot subframe classification tasks. namely, the pre-trained model can predict the political bias of articles with previously unseen topics and predict subframe groups with a limited quantity of labeled data. this greatly reduces the 1 code and data available at https://github.com/ msu-nlp-css/close_framing . ## related work framing is a powerful political strategy that is used to influence public opinion. hence identifying what and how frames are used is a critical task in political communications. framing in news media and social networks has been studied to analyze political polarization @xcite @xcite . however, annotating data for framing is challenging due to the nuanced language of frames across political issues. to overcome this challenge, computational social scientists follow a topic-specific codebook to manually annotate documents with frames @xcite @xcite . the most commonly used codebook is the policy frames codebook @xcite , which proposes a general coding scheme for fifteen high-level frames across policy issues. based on this codebook, @xcite collected and released the media frames corpus (mfc). the mfc contains more than 20,000 news articles on three policy issues: immigration, smoking, and same-sex marriage. following the release of mfc, the first largescale open-source frames dataset, many researchers studied and analyzed frames by leveraging the mfc. @xcite use political tweets to extract phrases that frequently appear in each of the frames and propose a framing detection model that uses both linguistic features and the extracted ideological phrases. @xcite extend the u.s. policy frames to russian policy frames and analyze 13 years of russian news articles. they derive the lexicons of each frame from the mfc and translate them to russian to generate russian framing lexicons. huguet @xcite propose a multi-task model that learns metaphor, framing, and emotion in political discourse and uses the mfc to predict frames. although our method also jointly models political bias and framing, our approach differs in that the main task of our model is to embed language used in subframes with contrastive learning. additional research has narrowed down the general policy frames to suggest issue-specific frames. @xcite add twitter-specific frames to the policy frames and annotate and analyze the tweets of u.s. politicians. roy and goldwasser (2020) build topic-specific lexicons, which they de-fine as subframe indicators, by using an embedding model to generalize the mfc lexicon for analyzing media bias. we utilize their subframe indicators to create our framing triplet dataset. our proposed model also includes an embedding model that generalizes text containing subframe indicators. however, we integrate political bias information into our model directly by adding bias classification as an auxiliary task. other approaches for detecting frames are based on topic modeling algorithms. latent topics of a given corpus are extracted with a topic modeling algorithm, often latent dirichlet allocation (lda) @xcite . lda yields statistically significant words of each topic, which are used as candidate indicators for defining frames. @xcite provide an open-sourced tool for analyzing frames in multilingual texts. given text inputs, their web-based system sends the lda topic modeling output to a user's email so that the user can decide and label frames on the given topics. however, we note that the output of lda is a list of keywords in each topic, not frame. the topic-based words are helpful guidance in framing annotations and exploratory analysis but are not appropriate data to be used for supervised framing analysis. ## framing triplet dataset we introduce the framing triplet dataset, which contains 25,627 news articles on three politically polarized topics: abortion, gun control, and immigration. we extend the hyper-partisan news dataset @xcite and the dataset by @xcite , which consists of news articles on the three topics until 2019. open-source crawlers pygooglenew and news-please were used to collect recent articles from 2020 to 2022. we query the google news rss feed with keywords of each topic and crawl the articles. the keywords are "abortion" for the abortion data, "gun" for the gun control data, and "immigrant" and "immigration" for the immigration data. then we assign the political bias label of each article according to media bias fact check foot_0 , the largest crowdsourced media bias resource on more than 4,500 media sources and journalists. the media bias fact check categorizes each media source into one of the following five biases: "left bias," "leftcenter bias," "least bias," "right-center bias," and "right bias." for our study, given the partisan divide of current u.s. politics, we only consider articles from "left bias" and "right bias" media. in total, our dataset includes 12,911 left-biased articles and 12,761 right-biased articles. the detailed statistics of the dataset can be found in after labeling the political bias of the collected articles, we extract headlines and sentences in articles that contain subframe indicators. the subframe indicators, suggested by @xcite , are the issue-specific subclassifications of the policy frames by @xcite . these subframe indicators are collected via the following three steps. first, the top-250 unigrams for each of the 15 policy frames are gathered based on their pointwise mutual information (pmi) scores @xcite . then, each paragraph in the article is annotated with the policy frames based on the number of unigram matches. finally, repeating phrases in the annotated paragraphs are grouped to form subframes, which represent topicspecific lexicons. they defined 20 subframes for the topic of abortion, 22 subframes for the topic of immigration, and 19 subframes for the topic of gun control. the list of subframes can be found in appendix a. we refer to @xcite for the full list of subframe indicators. finally, we construct triplets with the extracted sentences. given an anchor sentence s a , we define its positive sample s p as a sentence with subframes from the same subframe group and its negative sample s n as a sentence with subframes from a different subframe group. for example, the sentence "the first backlash to the roe decision came primarily from groups representing u.s. catholics." contains a subframe indicator "roe decision" that belongs to the subframe "roe v. wade." its positive sample (s p ) should be a sentence with a subframe indicator of that same subframe, "roe v. wade." its negative example (s n ) will be a sentence with a subframe indicator from a different, random subframe group such as "birth control." the triplet is formed as (s a , s p , s n ). an example of the triplet data is shown in figure 2 . ## methodology we propose close: a multi-task learning model that jointly learns to embed sentence framing language and predict political bias. as shown in figure 1 , the model consists of a bert-based or roberta-based encoder, followed by a pooling layer that generates a sentence embedding, which feeds into a classifier for political bias prediction. we adopt a contrastive learning objective so that a sentence with subframe indicators close to other sentences of the same subframe and far from those of different subframes will be similarly reflected in the embedding space. embedding via pooling. given a sentence s of tokens {t 1 , • • • , t n }, we embed the sentence using bert or roberta to get a sequence of token embeddings {e(t 1 ), • • • , e(t n )}. then a pooling operation is applied to the token embeddings to derive a fixed size sentence embedding e(s). there are three possible pooling strategies: (1) taking the output of the cls token, (2) computing the mean of all output vectors, and (3) computing a max-overtime of the output vectors. close employs the default pooling operation of mean strategy, which has shown the best experimental results on semantic textual similarity and natural language inference tasks @xcite . thus, the final sentence embedding is e(s) = 1 n n @xmath0. learn subframes with a contrastive learning objective. we apply a contrastive learning objective to capture the framing embedding space. to do so, we fine-tune the bert-based (or robertabased) encoder to encourage embeddings that will fall within the same subframe group to be closer and the embeddings within a different subframe group to be more distant from a given anchor sentence's embedding. every anchor sentence s a has a positive sentence s p and a negative sentence s n , corresponding to the triplets within the framing triplet dataset. for example, as shown in figure 2 , the anchor sentence has a subframe indicator "roe decision" (shown in bold), which belongs to the subframe group "roe v. wade." while its positive sentence has subframe indicators "overturns roe" and "roe v. wade" that belong to the same subframe group, the negative sentence has a subframe indicator "march for life" that belongs to a different subframe group "pro-life." given a triplet of sentences (s a , s p , s n ), the contrastive learning loss is computed as follows: predict political bias. the proposed model adds a binary classifier to the output of the bert-based encoder to predict the political bias of an input (anchor) sentence. the sentence embedding of this anchor sentence is given as an input to the classifier, which predicts whether the sentence is from a liberal (left) or conservative (right) news source. given an anchor sentence s i , we predict a stance label y @xmath1, 1}. we use binary crossentropy as the loss function: jointly learn framing and political bias. finally, we combine the learned losses in order to build an embedding space that can group subframes as well as separate the contexts of opposing political stances, i.e., distinguish between liberal and conservative biased news. the final loss that computes the weighted sum of the contrastive learning loss l c and the classification loss l bce is: ## experimental setup we evaluate the performance of our proposed method on the tasks of (1) political bias classification and (2) subframe detection. for the first task, a model makes a binary prediction of whether a given text has a liberal or conservative perspective. we compare the classification performance of our proposed model to the baseline models. further, an ablation study on the classification objective is performed, and the classification performance of the pre-trained models on unseen topic data is studied. as for the second task, we test the performance of subframe classification and analyze the effect of the bias classification task on the subframe classification. finally, we propose a novel topic modeling method for the output embeddings and show experimental results. ## conclusion we have presented close, a framework that incorporates subframes and political bias using contrastive learning and multi-task learning. our proposed joint objective outperforms baseline models in predicting subframes and political bias. further, our pre-trained model adapts to zero-shot learning of political bias and few-shot learning of subframes. finally, we propose a topic modeling method for the subframe embeddings and extract a list of keywords, which can be helpful for future subframe extensions and annotations. we plan to extend this work to embed general policy frames for study in unsupervised settings.