Catch Me If You Can? Not Yet: LLM s Still Struggle to Imitate the Implicit Writing Styles of Everyday Authors (2025)

Abstract: As large language models (LLMs) become increasingly integrated into personal writing tools, a critical question arises: can LLMs faithfully imitate an individual’s writing style from just a few examples? Personal style is often subtle and implicit, making it difficult to specify through prompts yet essential for user-aligned generation. This work presents a comprehensive evaluation of state-of-the-art LLMs’ ability to mimic personal writing styles via in-context learning from a small number of user-authored samples. We introduce an ensemble of complementary metrics—including authorship attribution, authorship verification, style matching, and AI detection—to robustly assess style imitation. Our evaluation spans over 40,000 generations per model across domains such as news, email, forums, and blogs, covering writing samples from more than 400 real-world authors. Results show that while LLMs can approximate user styles in structured formats like news and email, they struggle with nuanced, informal writing in blogs and forums. Further analysis on various prompting strategies such as number of demonstrations reveal key limitations in effective personalization. Our findings highlight a fundamental gap in personalized LLM adaptation and the need for improved techniques to support implicit, style-consistent generation. To aid future research and for reproducibility, we open-source our data and code.

## introduction every time humans write, they leave behind a unique linguistic fingerprint @xcite , a subtle and subconscious reflection of their personal style. to enhance their writing, humans have been using writing tools from spell and grammar checkers to paraphrasers for quite some time @xcite . but, the rise of large language models (llms) has introduced a powerful, all-in-one assistant capable of drafting, editing, and rephrasing text @xcite . despite their impressive fluency, llms often default to a generic style learned from vast web data, stripping away the personal touch that makes writing feel authentic (padmakumar and he, 2024). this raises concerns about authorship dilution and being flagged as ai-generated content @xcite . to counter these concerns, some systems (e.g., claude ai foot_0 ) offer configurable parameters like tone, voice, and formality. yet these controls fall short as users' personal styles are nuanced and rarely reducible to a few sliders @xcite . a more natural solution is to guide the model with examples of an individual's prior writing, employing few-shot prompting to emulate an individual's style more faithfully, like the task shown in figure 1 . while llms have shown promise in mimicking public figures @xcite and fictional characters @xcite whose extensive data footprints on the internet make them easy to model, their ability to replicate the style of everyday users remains largely unexplored. can llms truly adapt to the personal style of an everyday author with only a handful of casual interactions and no explicitly defined stylistic identities? existing research on personal-style generation for ordinary users is limited @xcite , and even more pressing is the lack of robust evaluation: how do we determine if a generated text from an llm can belong to the same individual in style and voice? while the concept of authorship is inherently subjective,computational methods like authorship attribution (aa) and authorship verification (av)-well-established in forensic linguistics-offer promising proxies. the theory of linguistic individuality @xcite suggests that a person's writing style often forms a consistent style model. if llms can generate text that fits within this style model while being recognized as from the same author by the two computational authorship models, it brings us closer to achieving truly personalized text generation. in this work, we examine whether llms can imitate the implicit writing styles of everyday users through in-context learning, without access to explicit stylistic instructions, via the generation task shown in figure 1 . our investigation is motivated by and rooted in a realistic usage scenario: most users interact with llms via standard interfaces rather than custom plug-ins or fine-tuned pipelines @xcite . while methods like personalized fine-tuning @xcite or advanced prompting strategies @xcite can improve stylistic alignment, they are often impractical in time-sensitive, real-world settings. instead, we focus on evaluating the out-of-the-box ability of current llms to replicate personal writing style by conditioning on a few user-authored samples, reflecting the kind of implicit personalization that is most accessible and scalable in practice. to this end, we conduct a comprehensive evaluation of state-of-the-art llms on the task of personalized writing imitation, defined as emulating an individual's implicit writing style solely from prior writing samples, without explicit stylistic instructions and semantically conditioned on a content summary (see figure 1 ). given the inherent difficulty of measuring personal style, we adopt a diverse suite of computational evaluators, including authorship attribution, authorship verification, linguistically grounded style metrics, and state-ofthe-art ai detection tool. this ensemble approach enables a robust and multifaceted assessment of stylistic fidelity and human-like generation. using this framework, we evaluate frontier models from leading providers, including openai, google, meta, and deepseek, across writing domains such as news articles, emails, online forums, and personal blogs. these domains reflect a range of everyday writing scenarios with distinct stylistic constraints. our datasets span over 400 authors and build on prior work to ensure diversity and realism. we investigate whether few-shot demonstrations of a user's prior writing are sufficient for llms to generate outputs that match the author's voice and style under our evaluators. results show that while llms can partially emulate user style in more structured formats like news and email, they struggle with nuanced, informal expression in domains such as blogs and forums. generated outputs often default to an average, generic tone and remain readily detectable as ai-written. moreover, increasing the number of demonstrations offers limited gains in stylistic alignment. these findings underscore the current limitations of in-context personalization and highlight the need for more effective methods to achieve truly personalized generation, an essential step toward democratizing llm-based writing tools. ## related work personalization in llms spans a wide spectrum, from persona emulation @xcite to personalized content recommendations @xcite . in this study, however, we specifically focus on the subtle challenge of implicit personalized writing imitation, whether llms can generate text that reflects the inherent implicit style learned from an individual's prior writings. to situate our work, we review progress at the intersection of llmbased personalized writing and authorship analysis around llm generated texts. llms for personalized writings recent advances in llms have enabled impressive performance in general text style transfer @xcite , with a growing interest in personalized text generation @xcite . much of prior work focuses on replicating the style of wellknown figures, such as authors @xcite , celebrities @xcite , or fictional characters @xcite , where models benefit from ample training data and rich backstories. however, research on replicating the style of ordinary individuals remains underexplored, especially in terms of a rigorous and systematic evaluation. some recent studies have attempted individualized generation using retrieval-augmented generation (rag) systems @xcite or advanced in-context prompting strategies like trial-error-explain @xcite . yet, these approaches are often costly, impractical for real-time personalization, or dependent on repeated llm queries. crucially, these studies usually rely on an llm-as-a-judge approach @xcite to measure stylistic alignment. while llms have shown remarkable performance in authorship tasks @xcite , their judgment is still subjective and might contain bias for their generation @xcite . other evaluations have used metrics like rouge or meteor @xcite , which assess content overlap but overlook stylistic fidelity. authorship attribution (aa) has been used @xcite , but typically in narrow domains or limited to single-text completions without broader style considerations. therefore, our study introduces a comprehensive evaluation framework grounded in authorship and stylometric analysis, aiming to systematically assess whether llms can truly replicate an individual's writing style authorship analysis tasks to evaluate whether llms can truly replicate individual writing styles, we draw from established authorship analysis tasks. authorship attribution (aa) identifies the author of a given text from a set of candidate authors @xcite , while authorship verification (av) determines whether two texts share the same author @xcite . these tasks traditionally rely on features such as n-grams and stylometric cues, e.g., lexical, syntactic, semantic, and structural patterns @xcite @xcite . while classic ml and dl models have long been used for stylometry, fine-tuned transformer models now represent state-of-the-art aa and av tasks @xcite . recently, stylometric analysis has also been employed to detect llmgenerated text @xcite @xcite , offering a promising path to evaluate whether llm outputs align with a target individual's writing style. ## methodology this study evaluates whether llms can mimic a user's implicit writing style using only a few prior examples without any explicit stylistic instructions. we consider a realistic scenario, as visualized in figure 1 , where users provide a handful of writing samples and a content summary, reflecting typical few-shot prompting, without describing their style in detail (which may also be challenging if feasible). this setup requires no model customization and offers a scalable path to personalization. while writing styles can evolve, a phenomenon studied in stylochronometry @xcite , they primarily focus on long-term changes in the writing of novelists or public figures @xcite . in our context, we assume that individuals tend to maintain a relatively stable style in everyday writing, making this assumption both practical and justified. figure 2 provides an overall visual illustration of our methodology with the rest of the section structured as follows. section 3.1 defines the task setup, 3.2 presents the evaluation framework, 3.3 describes the datasets, and 3.4 discusses our design choices and limitations. ## experimental setup in this section, we outline the key experimental choices of our study, with further details provided in appendix b. writing example sampling for each test sample, we provide 5 writing examples randomly sampled from the related train set written by the same author, shared across llms for stylistic mimicry. we employ random sampling since in principle we have no control over writing examples a user may include for llm prompting. this ensures a more realistic evaluation. test samples summarization we prompt gpt-4.1 (openai, 2025) to summarize each test sample. baseline as a baseline, we prompt llms with the same test samples zero-shot (with only a content summary). as it is more or less expected that llms would perform worse in the style imitation task when not exposed to implicit personal writing style from provided writing examples than with ones. we use this simple baseline to (1) empirically validate this expectation and (2) to sanity check our evaluation framework proposed in section 3. as a proof of concept, we only run four llms zero-shot, excluding deepseek and llama models. evaluation models we train aa and av models using transformer-based encoders, specifically longformer-base-4096 and modernbert-base (see appendix b for details). for each author, a style model based on liwc @xcite and writeprint @xcite feature sets is also constructed by extracting distributional stylistic features from their writing samples. we use these three models to evaluate the stylistic fidelity of both human-written (see right below) and llm-generated texts (see the following sections). in addition, we employ gptzero as the ai detection tool introduced in section 3.2 off the shelf. ## results we present a comprehensive analysis of how effectively llms mimic personalized style under our evaluation framework. we also report semantic similarity with the original texts to confirm that generated outputs follow the given content while preserving stylistic cues from the examples. 4 additionally, an xgboost classifier trained on these feature representations achieves substantial top-1 authorship attribution accuracy: 46.16% (ccat50), 43.12% (enron), 36.04% (reddit), and 34.11% (blog), despite the high number of authors (50-150). these results demonstrate the strength of our stylometric framework in capturing individual writing styles. ## follow-up studies everyday users may go beyond providing random examples and a content summary to achieve more personalized writing. rather than relying on ran- ## conclusion this paper presents a comprehensive evaluation of state-of-the-art llms on their ability to mimic the implicit writing styles of everyday users through few-shot in-context learning. by combining authorship attribution, verification, stylometric modeling, and ai generation detection, across four diverse datasets, we provide strong empirical evidence that, despite improvements from exemplarbased prompting, current llms still struggle to reproduce nuanced personal styles-especially in informal and stylistically diverse domains. our analysis further shows that prompt design choices, such as length alignment and content similarity, moderately affect stylistic fidelity, but do not close the personalization gap. these findings highlight fundamental limitations in the stylistic adaptability of llms and suggest that achieving truly personalized generation remains an open challenge. future work should explore richer personalization signals and hybrid prompting and/or finetuning strategies to better capture the subtleties of individual writing styles in real-world settings.