Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Prompting (2025)

Abstract: Fine-grained personas have recently been used for generating ‘diverse’ synthetic data for pre-training and supervised fine-tuning of Large Language Models (LLMs). In this work, we measure the diversity of persona-driven synthetically generated prompts and responses with a suite of lexical diversity and redundancy metrics. First, we find that synthetic prompts/instructions are significantly less diverse than human-written ones. Next, we sample responses from LLMs of different sizes with fine-grained and coarse persona descriptions to investigate how much fine-grained detail in persona descriptions contribute to generated text diversity. Our results indicate that persona prompting produces higher lexical diversity than prompting without personas, particularly in larger models. In contrast, adding fine-grained persona details yields minimal gains in diversity compared to simply specifying a length cutoff in the prompt.

## introduction synthetic data generated from llms or other algorithms are increasingly used in pre-training and post-training recipes for llms @xcite . however, care must be taken to incorporate high-quality and diverse synthetic data at scale to avoid 'model collapse' @xcite . recently, persona-driven synthetic pipelines have been proposed to generate diverse synthetic prompts and responses from llms, often with very specific, fine-grained personas @xcite . however, does fine-grained persona prompting actually lead to increased 'diversity'? in this work we define and measure @xcite ) lexical diversity of llm responses with and without persona prompting towards answering this question. persona prompting @xcite , i.e. instructing an llm to respond to interactions from what are the best day trips near san francisco? for a quick escape from san francisco, consider these top day trips: napa valley for wine tasting, muir woods for redwood hikes, point reyes for coastal charm @xcite for influencers visiting san francisco, top day-trip spots with great content potential include: napa valley-stunning vineyards, luxury experiences. point reyes-breathtaking coastal views. figure 1 : we measure the lexical diversity of llm responses to prompts with no persona, a fine-grained persona, and its coarse analog. fine-grained personas do not improve lexical diversity noticeably. the perspective of an individual, demographic, or social group (described by a short textual description), has risen as a community standard for steering llm responses @xcite , enabling personalized interactions @xcite , and simulating human/group behavior towards answering scientific questions in psychology and social science @xcite . while there has been some evidence showing that llm performance on tasks in some domains improves with persona prompting @xcite , the results are inconclusive @xcite , and persona-driven prompting has been shown to misportray, flatten, and essentialize identities, and is susceptible to caricature and stereotypying @xcite @xcite . in this work, we examine the diversity claims proposed by persona-driven synthetic data pipelines. we use a suite of diversity metrics @xcite to measure the lexical diversity and redundancy in synthetic prompts and responses. we aim to answer two questions: 1. does prompting with personas lead to increased diversity in llm responses for the same instruction? 2. do finegrained persona descriptions lead to more diverse responses than less detailed (coarse) personas? figure 1 shows an example and overview of our experimental setup towards answering our research questions. in §3, we show that synthesized prompts from personahub are noticeably less diverse across all our metrics against comparable human-written/annotated prompts. we report our main findings in §5: persona-prompting does lead to higher diversity (over human responses) but only with larger model sizes, and coarse persona descriptions lead to text that is just as diverse as fine-grained descriptions. we release our code and model generations online at github.com/gaurikambhatla/ persona-prompting-diversity. ## related work persona-driven data synthesis personas have been used as a means to generate synthetic training datasets @xcite . the personas themselves are generated with llms at scale, with synthetic data generated by prompting llms to write texts across genres and domains from the perspective of a persona. recently, sethi et al. (2025) and venkit et al. (2025) investigate the lexical diversity of persona descriptions, and riaz et al. (2025) explore the lexical and content diversity of synthetic data in the biomedical and finance domain -our paper is the first to investigate if fine-grained detail in persona descriptions during prompting leads to improved diversity in model responses. text diversity diversity is an inherently subjective and value-laded metric to measure @xcite . however, we can identify some qualities of diversity that are desirable in synthetic data from llms and measurable with automated metrics: less repetition, fewer surface-level patterns, and less redundancy. @xcite validate a wide range of automated metrics to measure lexical diversity of text, which we use to measure diversity of llm responses in our paper. while we might expect fine-grained persona prompting to lead to improved scores on our chosen metrics over nopersona and coarse persona prompting, we find this not to be the case in §5. readability in addition to measuring lexical diversity and redundancy with the aforementioned metrics, we also evaluated two readability metrics on llm responses @xcite . we hypothesize that persona-prompting should lead to larger variations in reading level predictions on synthesized texts, since different persona descriptions should mirror the diverse reading levels of the individuals/groups they represent. ## experiments if persona-driven data synthesis with fine-grained synthetic personas leads to more diverse synthetic data, then we should expect: 1. improved diversity metrics where models are instructed to answer prompts with various personas. 2. fine-grained personas should have better diversity metrics over coarse, less detailed personas. data to test these hypotheses, we sample 100 prompts from dolly's creative-writing subset, as well as 100 fine-grained personas from person-ahub. we derive 'coarse' personas from person-ahub's fine-grained personas by simply extracting the first clause in the persona using stanza @xcite due to the consistent structure of fine-grained persona descriptions. for example, the fine-grained persona a pr manager with insights into public relations strategy and press releases for influencers corresponds to the 'coarse' persona a pr manager. conditions we evaluate our diversity metrics on model responses under the following conditions: fine-grained persona prompting seems to lead to increased variance in readability metrics-however the increased variances are not statistically significant (with levene's test @xcite , @xmath0.05 for all differences). overall, we find that personaprompting does improve lexical diversity for larger models, but the improvements are not practically significant. as previously noted, an explicit length cutoff yields a far more noticeable improvement in lexical diversity across all metrics. ## conclusion persona-prompting with fine-grained synthetic personas has been claimed to lead to 'diverse' synthetic data without adequately defining or measuring diversity. in this work, we measure lexical diversity and redundancy of synthetic prompts and responses generated with personas using a suite of diversity metrics. we find that persona-derived prompts are uniformly less diverse than humanwritten counterparts. when evaluated on creative writing prompts, persona-driven synthesis does lead to greater diversity scores, but only for the largest deepseek-v3 model. further, a simple, explicit length cutoff in the prompt yields a far more noticeable improvement in lexical diversity across all metrics unlike fine-grained detail in persona descriptions. our results add color and quantitative measurements to the claims of diverse synthetic data with fine-grained persona prompting, pointing to a recurring pitfall in overemphasizing the importance of fine-grained detail @xcite . in future work, we hope to continue exploring the question of content diversity of synthetic text derived from persona-driven prompting.