Improving Language Model Personas via Rationalization with Psychological Scaffolds (2025)

Abstract: Language models prompted with a user description or persona have been used to predict the user’s preferences and opinions. However, existing approaches to building personas mostly rely on a user’s demographic attributes and/or prior judgments, but not on any underlying reasoning behind a user’s judgments. We introduce PB&J (Psychology of Behavior and Judgments), a framework that improves LM personas by incorporating potential rationales for why the user could have made a certain judgment. Our rationales are generated by a language model to explicitly reason about a user’s behavior on the basis of their experiences, personality traits, or beliefs. Our method employs psychological scaffolds: structured frameworks such as the Big 5 Personality Traits or Primal World Beliefs to help ground the generated rationales in existing theories. Experiments on public opinion and movie preference prediction tasks demonstrate that language model personas augmented with PB&J rationales consistently outperform personas conditioned only on user demographics and / or judgments, including those that use a model’s default chain-of-thought, which is not grounded in psychological theories. Additionally, our PB&J personas perform competitively with those using human-written rationales, suggesting the potential value of synthetic rationales guided by existing theories.

## introduction recent advances in language modeling for user persona (i.e, characteristics, preferences, and behavior of a user) offer straightforward yet powerful ways to predict user behavior and decision-making (d @xcite . personas have enabled simulation capabilities in lms, such as helping survey design by simulating a wide range of human responses @xcite @xcite , simulating communities to study behavior @xcite @xcite , and producing diverse and largescale synthetic data @xcite . yet, simulated lm personas still struggle to align well with intended user behavior @xcite . a typical task for evaluating lm personas is opinion prediction: how accurately does the lm persona reflect the user's real opinion to questions such as "for your job or career aspects of your life, would you say that you are where you expected to be at this point in your life?" fine-tuning lms on user conversations and interaction history to create personas does not scale well due to the lack of adequate user-specific data @xcite @xcite . zeroshot prompting approaches, where a persona is specified using demographic attributes in a system prompt, avoid these issues but often produce inconsistent responses and expose known biases in current lms @xcite @xcite . to improve zero-shot prompting, recent approaches incorporate user judgments (prior user responses and history) in the prompt @xcite . for example, a user judgment like "i'm not very focused on my professional life right now" could inform their response to questions about their career expectations. while these methods provide additional context, they still fail to address a critical gap: the ability to rationalize why a user might have a specific judgment, an essential component to understand behavior. in our example, one plausible rationale for the user's judgment could be that they are prioritizing their family over their career at this time. in this work, we hypothesize that incorporating plausible post-hoc rationales for a user's judgment could improve lm personalization, bridging the gap between surface-level personas and deeper, more nuanced simulated behavior. however, collecting such rationales is expensive and existing datasets for building personas only contain judgments @xcite , and in rare cases, user demographics @xcite , but no rationales for judgments. to address this limitation, we introduce pb&j (psychology of behavior and judgments), a framework that uses post-hoc plausible, yet synthetic, lm-generated rationales for user judgments, to improve lm personas. the rationales for user judgments in our method stem from psychological theories, such as personality traits (e.g., big 5 traits (goldberg, 1993), schwartz's value theory @xcite ) and belief systems (e.g., primal world beliefs (clifton et al., 2019)); we call these psychological scaffolds. each psychological scaffold offers additional context to produce one potential (albeit generated) path of reasoning to support a given user judgment. this is in contrast to chain-of-thought rationales @xcite where no explicit psychological scaffold is employed; the lm relies on its own world knowledge to rationalize a test input prediction ad hoc. it is important to note that while our synthetic rationales may not reflect the true reasoning behind a user's judgment, they may contain plausible markers of real behavioral reasoning due to the presence of such reasoning in the lm's training data @xcite @xcite . indeed, humans use everyday reasoning to rationalize behavior; this is known as folk psychology @xcite @xcite . given a user's demographics and prior judgments, pb&j produces lm-generated rationales for user's judgments to construct a richer, more comprehensive user description for persona prompting. our experiments show that when prompted with such richer descriptions, lms can result in more accurate personas, as evaluated on a set of test instances. we evaluate pb&j on two tasks that represent different aspects of personalization: opinion prediction in opinionqa @xcite and movie preference prediction in movielens (harper and @xcite . on both benchmarks, we show that lm personas prompted with pb&j rationales are significantly better at predicting a user's response to test instances than those prompted with user demographics and/or prior judgments alone, including those with additional chain-of-thought reasoning. improvements from pb&j are seen across nearly all demographic categories, and pb&j can improve personas with or without demographic information. finally, we present a pilot experiment where we collect humanwritten rationales, along with their demographics and judgments on a subset of opinionqa; lmrationales nearly match the performance of humanwritten rationales in pb&j, showing the efficacy of rationales generated with our approach. at a high level, our work highlights the value of synthesizing data with the appropriate structural context to produce plausible rationalizations that improve the modeling of user opinions and preferences in capable language models. ## background humans commonly use explanations to rationalize each other's actions @xcite . we hypothesize that rationales are common enough in our discourse (e.g., narrative text of novels, biographies, social media) to be prevalent in training corpora for lms to generate effective rationales in zero-shot settings @xcite . in order to better guide lms, we employ various psychological scaffolds, which are templates based on more structured theories of psychology, to rationalize user judgment. ## lm personas using pb&j consider a task where an lm has to adopt the persona of a given user, and perform a task (e.g. answer opinion-based questions) as their proxy. in this section, we describe how we leverage lm rationalization to construct lm personas, which are used as prompts to generate personalized responses as a proxy for the user. we begin by setting up a basic persona description composed of user demographics and augmented with a set of user judgments, just as in previous work @xcite . we then present pb&j, which enriches persona descriptions by providing additional context in the form of rationales, and demonstrate how pb&j is used to generate personalized responses as a proxy for the user. ## experiments task and datasets. we conduct experiments on two datasets -opinionqa @xcite and @xcite . opin-ionqa is a collection of american public opinion surveys @xcite conducted by pew research containing user-selected answers to multiple-choice questions in 15 different topics ranging from food safety to guns, with rich accompanying demographic attributes. the task is to predict a user's answer for a question. for opin-ionqa, we use a subset of 750 users, and 10 test questions for each user. movielens contains timestamped movie ratings (between 1-5) corresponding to individual users, with gender, age, occupation and location of each user. the task is to predict a user's rating for a movie. we use a subset of 100 users and 10 test movies for each user. further details and splits of the dataset are presented in appendix c. example inputs-outputs of each of these datasets are shown in figure 2 . seed judgments. our persona description generates rationales for a set of seed judgments. for opinionqa, these seed judgments are a 'train' set of questions and users' corresponding answers to them. the questions and answers are then converted into declarative forms (e.g., if a user responds no, not really to "are you currently focused on your professional life and career?", the declarative form would be i'm not very focused on my professional life right now). for movielens, we use movie ratings provided by the user with earlier timestamps for predicting ratings by the same user with later timestamps. each judgment consists of a movie, its rating, and a short description of the movie consisting of its plot synopsis, actors, and directors. for our main experiments, a fixed set of 8 seed judgments are provided in the persona description. evaluation. we use a 'test' set of opinion questions and movies to evaluate the personas. since our personas are customized to each real user, we calculate accuracies and standard deviations for both opinionqa and movielens, macro-averaged for each user. language models. we use two lms of varying sizes -gpt-4 2 (openai et al., 2024) and mistral for the latter setting, we also include results using chain-of-thought reasoning @xcite as demographics + judgments cot , where lm t is prompted to reason about the test question using the persona information in order to arrive at the answer. this is lm t 's default rationale that is not explicitly grounded in any psychological scaffold. in section 3.1, only demograph-ics, only judgments and demographics + judgments correspond to q d , q j and q b respectively. ## using pb&j with human-written rationales ## discussion lm-generated rationales help even with a limited budget of judgments. unlike demographics, user judgments take time to collect, making it crucial to assess personas with limited data (section 4). as shown in figure 3 , pb&j with pri-mal world beliefs outperforms both baselines (only judgments and demographics + judgments) across varying number of judgments. even with just two judgments, pb&j far surpasses baselines, highlighting the value of lm-generated rationales in low-data settings. as the number of judgments increases, the performance of all methods improves. however, the rate of improvement for all diminishes after 8 judgments. this indicates that while all methods benefit from additional user judgments, pb&j maximizes its potential gains earlier due to the contextual richness provided by lm-generated rationales. groups rather than relying on gains from specific demographics. to assess this, we compare pb&j (with primal world beliefs scaffolds) to de-mographics + judgments across education, race, income, and gender in figure 4 (splits across all demographics in figure 5 ). pb&j consistently outperforms the baseline across all demographics. while gains are notable for users with postgraduate education and those identifying as asian or mixed race, improvements extend across all education levels, racial groups, income brackets, and genders. by incorporating scaffolded rationales, pb&j improves with diverse user perspectives, demonstrating broad effectiveness with synthetic rationales. ## conclusion this work introduces pb&j, a framework that improves lm personas by incorporating plausible, yet synthetic rationales to explain user judgments. by leveraging psychological scaffolds, pb&j improves lm persona accuracy across diverse opinion prediction and preference modeling tasks, while performing, at best, close to human-written rationales. additionally, pb&j remains effective even with a limited number of user judgments, highlighting its potential for real-world personalization applications, with scarce user history.