“Was it “stated” or was it “claimed”?: How linguistic bias affects generative language models (2021)

Abstract: People use language in subtle and nuanced ways to convey their beliefs. For instance, saying claimed instead of said casts doubt on the truthfulness of the underlying proposition, thus representing the author’s opinion on the matter. Several works have identified such linguistic classes of words that occur frequently in natural language text and are bias-inducing by virtue of their framing effects. In this paper, we test whether generative language models (including GPT-2 (CITATION) are sensitive to these linguistic framing effects. In particular, we test whether prompts that contain linguistic markers of author bias (e.g., hedges, implicatives, subjective intensifiers, assertives) influence the distribution of the generated text. Although these framing effects are subtle and stylistic, we find evidence that they lead to measurable style and topic differences in the generated text, leading to language that is, on average, more polarised and more skewed towards controversial entities and events.

## introduction with subtle changes in word choice, a writer can influence a reader's perspective on a matter in many ways @xcite . for example, ## discussion through our experiments, we see that language models indeed respond differently when given texts that show markers of opinion bias, manifesting in both topical and stylistic differences in the language generated. this finding has both positive model top-weighted named entities 124m israel (24.1), gaza (22.15), muslim (21.5), christ (21.13), korea (24.36), russia (22.33), north (21.81), us (21.71) 355m israel (22.5), jews (21.02), serbia (20.93), trump (20.9) padres (22.13), national (20.88), junior (20.69), tm (20.48) 774m mwa (30.79), trump (21.45), rabbi (19.94), god (19.55) duke (21.63), scot (20.51), obama (19.74), yoga (19.45) 1.5b trump (18.6), kosovo (18.4), pakistan (17.8), muslim (17.82 buckley (21.04), tm (20.53), lott (19.23), ireland (18.99) and negative implications. the positive is that differentiating such subtle aspects of language requires sophisticated linguistic representations; if models were indifferent to the types of edits made in the sentences we study here, it would suggest a failure to encode important aspects of language's expressivity. the negative implication is that, when deployed in production, it is important to know how language models might respond to prompts, and the demonstrated sensitivity-which may lead models to generate more polarized language and/or trend toward potentially sensitive topics-can be risky in user-facing applications. the trends observed here also suggest potential means for intervening to better control the types of generations produced by a model. for example, if linguistic bias cues are used unintentionally by innocent users, it might be possible to use paraphrasing techniques to reduce the risk of harmful unintended effects in the model's output. in contrast, if such linguistic cues are used adversarially, e.g., with the goal of priming the model to produce misleading or opinionated text, models that detect this implicit bias @xcite could be used to detect and deflect such behavior. the effect of model size we perform all analyses for every model ranging from 124m to 1.5b parameter gpt-2 models foot_2 . overall, we do not see significant correlations between the size of a model and its response to framing effects. importantly, we see that the observed behaviors arise even in the smallest model (124 million parameters), suggesting that it does not require particularly powerful models in order to encode associations between these linguistic cues and the larger topical and discourse contexts within which they tend to occur. ## investigating larger language models: a case study on gpt-3 post-acceptance, we were given access to gpt-3 @xcite , a language model that is similar in construction to the gpt-2 models, but is an order of magnitude larger, containing 175 billion parameters. we perform the same analysis described in prior sections and report results on the gpt-3 model here. specifically, for the same prompt pairs, we obtain generations of up to 300 words from the gpt-3 model, and we do this 3 times per sample for robustness. overall, the conclusions do not differ from those drawn using the smaller gpt-2 models. ## conclusion we investigate the extent to which framing effects influence the generations of pretrained language models. our findings show that models are susceptible to certain types of framing effects, often diverging into more polarised points-of-view when prompted with these. we analyse the semantic attributes, distribution of words, and topical nature of text generated from minimal-edit pairs of these types of linguistic bias. we show that cues of opinion bias can yield measurable differences in the style and content of generated text.