Viable Threat on News Reading: Generating Biased News Using Natural Language Models (2020)

Abstract: Recent advancements in natural language generation has raised serious concerns. High-performance language models are widely used for language generation tasks because they are able to produce fluent and meaningful sentences. These models are already being used to create fake news. They can also be exploited to generate biased news, which can then be used to attack news aggregators to change their reader’s behavior and influence their bias. In this paper, we use a threat model to demonstrate that the publicly available language models can reliably generate biased news content based on an input original news. We also show that a large number of high-quality biased news articles can be generated using controllable text generation. A subjective evaluation with 80 participants demonstrated that the generated biased news is generally fluent, and a bias evaluation with 24 participants demonstrated that the bias (left or right) is usually evident in the generated articles and can be easily identified.

## introduction natural language generation is defined as the creation of understandable text using a language model (lm) trained on a large collection of texts. an (lm) is a probability distribution over a sequence of words. given a set of training text sequences, we can train an lm to produce texts similar to the training data. researchers have used deep learning algorithms to generate more fluent and semantically meaningful texts than those generated using conventional methods like n-grams @xcite . such lms are being used to generate image captions @xcite , perform machine translations @xcite , paraphrase and summarize text @xcite . high performance lms can generate fake news, fake reviews, and fake comments @xcite . recent studies have revealed various types of bias in top us news sources, which often report political news in a biased way, for example, attention can be drawn to particular events and entities while ignoring others @xcite @xcite . the selection of what to report about an entity (positive or negative) produces bias. there are two major political sides in the u.s.: democrats on the left and republicans on the right. the news aggregating platforms like google news and yahoo news are the most viewed news websites in u.s. with 150 and 175 million unique visitors every month, respectively @xcite . they offer content relevant to a wide range of global audiences, and therefore, they have a responsibility to maintain the same sentiment and bias. however, they can utilize language models to generate biased content (news headlines and articles) to model the behavior of their readers. exposure to biased news is very harmful as it can increase/flip the political bias of a reader @xcite . for example, @xcite found that exposure to biased news can alter the political inclinations of people, and @xcite found that false representation of news from a news source can lead to broken trust between the reader and the news source. previous works on media bias mostly focused on detecting bias either by using cues from the social media presence of the news sources @xcite @xcite , or by analyzing how bias is manifested within each news article @xcite . @xcite focused on flipping the bias of news headlines, which is a short one line text. @xcite showed that exposure to opposing views can increase political polarization. to the best of our knowledge, ours is the first attempt at generating full length biased news articles using high performance language models. ## related work in this section, we discuss related work on political bias datasets, bias analysis, bias generation and detection in news articles. ## biased news generation the most important parts of the proposed method for generating biased news is the gpt-2 text gen-eration model @xcite and the controllable text generation model @xcite . as shown in figure 2 , we used a two step approach to generate biased news: generation and validation. in the generation step, an attacker provides an original news article x as the seed input to a generation models. the models then generate a modified article x' based on x. in the validation step, the generated articles are classified on the basis of bias. the attacker is assumed to have access to such a classifier and uses it to segregate left-and right-biased news. the details of our proposed method are discussed below. ## discussion our use of the api made available to us by "the bipartsan press" to evaluate bias is a major limitation of this study. evaluating text for bias is a very complex problem. the api was built on a roberta based model trained on a dataset curated by adfontes media. the dataset was annotated by 20 expert annotators with at least a college degree after an extensive screening and training process foot_5 . hiring and training such annotators is expensive, and relying on non-expert annotators to calculate media bias in generated news is not promising. since our findings conforms to the results reported by relevant literature on media bias, it is safe to assume that the results obtained using the roberta-based model (with a 4% error rate) are reliable in terms of segregating left-biased media from right-biased media. ## conclusion and future work we have presented a threat model and discussed how news aggregators (attackers) can manipulate readers' opinions by flipping or increasing their bias. we described two language models generating biased news: the high-performance gpt-2 lm and the grover lm for controllable text generation. we used a large news article dataset to fine tune them. we used a roberta-based regression model to create automatic bias ratings and to evaluate bias in generated news. subjective evaluation of generated news articles by 80 participants suggests that they made random selections between the machine-generated and human-written news excerpts, indicating that the machine-generated news is fluent and looks similar to human-written news. out of the 80 participants, 24 were chosen for a bias evaluation. the participants were able to see a clear bias most of the times, and marked correct bias 63% of the times. for future work, techniques for a more granular control on text generation can be explored, where one can adversarially inject bias to generate twisted versions of news stories. techniques to introduce bias during machine translation of a news article from one language to another can be explored and evaluated by comparing the generated news after translation with the news generated by non-native speakers while converting news from other languages. apart from named entities and sentence length, there are more intrinsic patterns representing presence of bias in text. exploration studies to find such patterns can also be done in future to better understand bias distribution in text. another future direction can be to quantify the impact of delivering biased news to real-world users using some social media platform.