Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM -Generated Text (2025)

Abstract: Persona-prompting is a growing strategy to steer LLMs toward simulating particular perspectives or linguistic styles through the lens of a specified identity. While this method is often used to personalize outputs, its impact on how LLMs represent social groups remains underexplored. In this paper, we investigate whether persona-prompting leads to different levels of linguistic abstraction—an established marker of stereotyping—when generating short texts linking socio-demographic categories with stereotypical or non-stereotypical attributes. Drawing on the Linguistic Expectancy Bias framework, we analyze outputs from six open-weight LLMs under three prompting conditions, comparing 11 persona-driven responses to those of a generic AI assistant. To support this analysis, we introduce Self-Stereo, a new dataset of self-reported stereotypes from Reddit. We measure abstraction through three metrics: concreteness, specificity, and negation. Our results highlight the limits of persona-prompting in modulating abstraction in language, confirming criticisms about the ecology of personas as representative of socio-demographic groups and raising concerns about the risk of propagating stereotypes even when seemingly evoking the voice of a marginalized groups.

## introduction large language models (llms) are increasingly used in numerous tasks involving the generation of an artifact (being a text, an image, or a video) by means of an input expressed in a natural language. a recent trend, influenced by the vision of llms as agents, is the use of persona-prompting as a strategy to personalize the generation and steering model behavior to assume specific viewpoints @xcite @xcite . this method assumes that the persona activates corresponding parametric knowledge within the llm, enabling it to simulate viewpoints or linguistic styles aligned with the assigned identity or character @xcite . while persona-prompting has shown promise in subjective tasks and style approximation @xcite , its overall effectiveness remains underexplored and debated @xcite . in this paper, we examine how persona prompts differ from a generic ai assistant when generating short texts about individuals tied to socio-demographic categories and potentially stereotypical attributes (e.g. "german" and "always on time"; see figure 1 ). this paper investigates the risks of personaprompting leading to stereotypical text generation, regardless of the associated attributes and the persona used in the prompt. stereotypes are reflected and transmitted by subtle linguistic cues indicating a high degree of category-generalization (beukeboom and burgers, 2019). following the linguistic expectancy bias (leb) framework @xcite , this category-generalization can be quantified by measuring the level of abstraction of a text: the more a text contains abstract language, the more it conveys a stereotypical (and biased) description of an individual or a group @xcite . we ad-dress this through the following research questions: rq1: do llms produce more abstract descriptions when a stereotypical attribute is paired with a socio-demographic category in the prompt? rq2: how does the level of abstraction in texts generated via persona-prompting compare to that of a generic, unconditioned ai assistant? rq3: does the abstraction level in personaprompted text change when the persona and the socio-demographic category belong to the same in-group? we approach these research questions by analysing the responses of six open-weight llms of different sizes (from 3b up to 72b parameters) when prompted to write texts in three different conditions given a socio-demographic characteristic of an individual and: (i.) an expected, stereotypical attribute (e.g., "german" -"always on time"); (ii.) the negated stereotypical attribute ("german" -"not always on time"); and (iii.) a random attribute ("german" -"loves fried chicken"). to quantify abstraction, we operationalize the leb framework by means of three dictionary-based measures that capture the concreteness, the specificity, and the negations in a text. with this framework, we model and quantify the abstraction level of the generated texts and compare them across 11 personas and the default ai assistant. our extensive experiments show that llms use equally abstract language when describing stereotypical and non-stereotypical category-attribute combinations, regardless of whether they are written by an ai-assistant or persona-conditioned model. while texts generated by personas differ from the ai assistant in terms of superficial wording, they remain generic and stereotyped, failing to reflect different socio-demographic perspectives. contributions: we introduce selfstero, a new dataset of self-reported stereotypes collected from reddit ( § 3). additionally, we show the inadequacies of closed-task-based evaluations for assessing the level of stereotypes in two families of openweight llms with varying sizes. in particular, we provide a detailed analysis of linguistic profile (measured in terms of concreteness, specificity, and negations) in llm-generated text, guided by the leb framework, across 11 persona prompts and two model families ( § 4 and § 5). ## task and abstraction metrics our primary goal is to evaluate whether personaprompting induces linguistic patterns that differ meaningfully from those of a generic ai assistant. to assess the extent of this variation, we have framed our task as a zero-shot generation task where models are prompted to write a short text given a socio-demographic category-attribute pair. linguistic patterns can vary along many dimensions. we focus on linguistic bias and operationalize it in a novel way. we follow the linguistic category model (lcm) @xcite , which defines bias in terms of differential use of abstraction. more specifically, we draw from the linguistic expectancy bias (leb) framework @xcite which describes the tendency to use more abstract language when behaviors align with stereotypical expectations, and more concrete language when they do not the lcm, however, relies on heavy manual coding based on word category information. moreover, the notion of abstraction itself is multifaceted and not consistently defined within lcm. abstraction can be understood in at least six ways, ranging from categorical knowledge derived from experience to schematic, memory-based representations @xcite . one key assumption in lcm is that "higher abstraction levels imply generalizations" @xcite ) about an individual or group. this suggests that lcm conflates abstraction with another variable, namely specificity, a relational concept that describes how general or specific a term is relative to another @xcite . additionally, abstraction in lcm may also implicitly refer to conceptual distance from physical experience, as in abstract versus concrete concepts @xcite . this is reflected in the importance of state verbs and nouns in the lcm's abstraction scale. considering this, we operationalize the lcm and the leb by means of three metrics: we approximate this as the number of neg labels in the parts-of-speech tagged texts normalized by the number of tokens. in our analysis, we do not report a unique measure of "abstraction" but we keep the three metrics separated since they complement each other and offer a more nuanced view of linguistic bias. in the next section, we introduce our dataset of human-written texts, which we use as a reference point. we compute the concreteness, specificity, and negation metrics on this corpus to establish baseline patterns of abstraction in natural language. these serve as the comparative foundation for evaluating the outputs of various open-weight llms (prompted with and without personas). ## the self-stereo corpus we introduce self-stereo, a new dataset of selfreported stereotypes in english linking sociodemographic categories and attributes. we scraped 867 top-level comments from the reddit post "what stereotype is 100% accurate about you?" from the subreddit r/askreddit using the praw api. foot_2 comments typically followed the pattern "i am [category] and i [attribute]", where "attribute" can be realized as a behavior, a physical or mental condition, or an attitude. three expert annotators foot_3 labeled text-spans corresponding to sociodemographic categories and attributes, achieving a krippendorff's α of 0.80 measured on a subset of 100 items. the annotation guidelines are reported in appendix a. one annotator then normalized semantically similar attributes (e.g., "can't drive", "not good at driving") into unified forms (e.g., "cannot drive"). hateful messages and obviously sarcastic ones have been excluded. the remaining set contains self-reported stereotypic generalizations. the final dataset is composed by 710 @xmath0,@xmath1. in we applied the three linguistic bias metrics to this corpus. the human-authored texts present the following average values: @xmath2.35 (@xmath3.52), @xmath4.09 (@xmath5.68), and @xmath6.01. concreteness and specificity scores are derived from the majority of words in each sentence (concreteness: ≈84% from all pos; specificity: ≈86% from nouns and ≈78% from adjectives), indicating that the method captures a broad vocabulary range while relying on existing resources. the scores indicate that texts in self-stereo are relatively concrete, quite generic, and with low presence of negation, confirming their stereotype-consistent nature in line with the expectation from lcm and leb. the low presence of negation clearly indicates minimal overt rejection of stereotypical expectations. these values provide an empirical grounding for comparison with llm-generated content, allowing us to assess whether and how machine-generated language deviates from naturally occurring human discourse in its use of abstraction and related stylistic features. we have also evaluated how these stereotypical associations emerge in llms (details are reported in appendix b). by relying on a series a closed-task settings, we asked our models to predict the appropriate category given a stereotypical attribute, both in the affirmative (i @xmath7, and, vice versa, to pre-dict the stereotypical attribute given the category (i am german and @xmath8. overall, token accuracy is very low (maximum 0.1), showing that these stereotypical associations do not emerge easily. expected stereotypical categories are mostly generated for nationality/place of origin (specifically canadian, british, american, and mexican) and race (mostly asian). we have also identified that, for @xmath9,@xmath10, llama32-3b refuses to answer more often when compared to the other models (total 173 cases over all versions and settings). 4 while at first, these results could be interpreted as lack of bias in the llms we have selected, they further confirm the criticism of closed-tasks to assess the presence of these phenomena @xcite . ## @xmath11,@xmath12 our experiments are designed to assess whether llms reproduce linguistic patterns associated with stereotype expressions, and whether such patterns shift under persona-based prompting. we probe this behavior by analyzing how models describe individuals based on a socio-demographic category and a given attribute, measuring the concreteness, specificity, and negation of the generated responses. the task is an open-ended text generation: given @xmath13, @xmath14, the model is asked to write a text. by systematically varying the type of attribute and the identity of the system prompt, we evaluate whether model generations exhibit systematic differences in the abstraction level of the texts as an indicator of linguistic bias. we design three experimental conditions: for each condition, we compare the texts produced by the generic assistant with those produced by each persona. this setup allows us to isolate the effect of persona-prompting on linguistic style and abstraction, and to evaluate whether adopting a persona amplifies, attenuates, or merely replicates the patterns found in the generic ai assistant. we test this setup on six open-weight llms of varying sizes and architectures, including llama32-3b, llama31-8b llama31-70b, qwen25-3b, qwen25-7b and qwen25-72b. for all models we used the instruction-tuned versions. all experiments have been run on four h100 nvidia gpus. comparison across models' sizes is essential to assess whether differences in abstraction vary with model scale, their sensitivity to stereotypes regardless of the experiment condition. ## results and discussion we structure the discussion of the results along three main blocks, each addressing one of the research questions we have presented in § 1, focusing on (i.) the contrast between stereotype-consistent and stereotype-inconsistent attribute pairings, and (ii.) the impact of persona-based prompting relative to a generic ai assistant. table e in appendix c). detailed results for each model are presented in table g in appendix d. ## conclusion in this work, we evaluated how persona-prompted llms change their language when describing a socio-demographic category and an associated attribute, whether the latter expresses a stereotype or not we operationalized a sociological framework and evaluated the degree of abstraction of gener-ated texts by llms. across several combinations of experimental conditions, models, and prompt strategies, we found that the generated texts are always mildly concrete, very generic, and with almost no negations. even when they describe a social category by taking a specific person (e.g. "let's meet alex . . . "), the overall degree of abstraction still leads to generalizations towards the entire category. this tells us that, independently of a superficial difference in the wording (measured via bleu and rouge-l), llms do not take the perspective that could make them generate a different text: whether an llm impersonates an anarchist or a conservative it will still end with a stereotypical, biased description. this occurs also when llms are prompted to generate texts about the same ingroup of the persona(s) they are taking. we have also observed that the qwen25 family of models is less sensitive to persona steering than lama3* one, suggesting that not all models are equally good at persona-prompting. while the study of biases and the perpetration of stereotypes in llms is a longstanding area of research in nlp, we believe that this type of analyses, grounded in social science frameworks, could be beneficial to further investigate their limits.