Flipping the Dialogue: Training and Evaluating User Language Models
Authors: Tarek Naous
Philippe Laban
Wei Xu
Jennifer Neville
Venue: ICLR 2026 Poster (2026)
Keywords: User Language Models
User Simulation
Interactive Evaluation
Post-Training
Primary area: foundation or frontier models, including LLMs
TL;DR: We introduce and evaluate user language models - models that are post-trained to simulate users that interact with assistants.

Abstract: Conversations with LMs involve two participants: a human user leading the conversation, and an LM assistant responding to the user's request. To satisfy this specific role, LMs are post-trained to be helpful assistants -- optimized to produce exhaustive and well-structured responses, free of ambiguity and grammar errors. User utterances, on the other hand, are rarely perfected, with each user phrasing requests in unique ways, sometimes putting in partial effort at each turn and refining on the fly. To evaluate LM performance in realistic settings, prior work simulated users in multi-turn conversations, often by prompting an LM originally trained to be a helpful assistant to act as a user. However, we show that assistant LMs make for poor user simulators, with the surprising finding that better assistants yield worse simulators. Instead, we introduce purpose-built User Language Models (User LMs) - models post-trained to simulate human users in multi-turn conversations. Through various evaluations, we show how User LMs align better with human behavior and achieve better simulation robustness than existing simulation methods. When leveraging User LMs to simulate coding and math conversations, the performance of a strong assistant (GPT-4o) drops from 74.6% to 57.4%, confirming that more realistic simulation environments lead to assistant struggles as they fail to cope with the nuances of users in multi-turn setups.

Published as a conference paper at ICLR 2026
FLIPPING THE DIALOGUE: TRAINING AND EVALUATING USER LANGUAGE MODELS
Tarek Naous Philippe Laban Wei Xu Jennifer Neville Microsoft Research, Georgia Institute of Technology tareknaous@gatech. edu plaban@microooff.. com
ABSTRACT
Conversations with LMs involve two participants: human user leading the con- versation, and an LM assistant responding to the user's request. To satisfy this specific role, LMs are post-trained to be helpful assistants optimized to produce exhaustive and well-structured responses, free of ambiguity and grammar errors. User utterances, on the other hand, are rarely perfected, with each user phrasing requests in unique ways, sometimes putting in partial effort at each turn and re- fining the fly. To evaluate LM performance in realistic settings, prior work simulated users in multi-turn conversations, often by prompting an LM originally trained to be a helpful assistant act as user. However, we show that assistant LMs make for poor user simulators, with the surprising finding that better assis- tants yield worse simulators. Instead, we introduce purpose-built User Language Models (User LMs) models post-trained to simulate human users in multi-turn conversations. Through various evaluations, we show how User LMs align bet- ter with human behavior and achieve better simulation robustness than existing simulation methods. When leveraging User LMs to simulate coding and math conversations, the performance of strong assistant (GPT-40) drops from 74.6% to 57.4%, confirming that more realistic simulation environments lead to assistant struggles as they fail to cope with the nuances of users in multi-turn setups.
mi niicroooftUserrLM-22
SIMULATING USERS IN CONVERSATIONS.
USING AN ASSISTANT LANGUAGE MODEL
USING A USER LANGUAGE MODEL
User Intent: Write Python function: given aray integers, ones betweon inclusive, reverse the array, and replece digits by their name from "One", "Two". "Three", etc.
GPT-40
UserLM-Bb
digits into names
Simulator Capability
ignore any number that is not the 1-to range
GPT-40 (Prompted)
Real Users
UserL_M-Bb (Trkkedd)
want the numbers they're between and
Diversity (1)
now add sorting into it
74.4
94.01
94.5
Dialogue Termination
want to flip the list after that
100
63.5
reverse the sorted list
Intent Decompostiion (1)
names for numbers
7.7
1.68
2.7
translate each remaining digit into its capitalized English name
from One to Nine
Naturainess (*)
3.3
90.15
80.2
Simple Direct User Turns
Robustress (t)
Nuanced & Indirect User Turns
Fallure
Success
54.9
100
94.3
Bonchmark Score: 74.6
Benchmark Scove: 57.4
Figure 1: Comparison of simulating users in conversations by prompting an assistant LM (GPT-4o) to roleplay user vs. our user language model UserLM-Sb (O+)) Both simulators converse with an assistant ($ GPT-4o) to solve a coding problem. The GPT-4o-based simulator produces simple and direct user turns, enabling the assistant to successfully solve the task. In contrast, UserLM- 8b reveals the intent in a correct but paraphrased form, leading the assistant to fail on the task. UserLM-8b is more aligned with the behavior of real users, helping better estimate the performance of assistants in realistic, multi-turn conversations.
Work done while interning Microsoft Research.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Simulating human users is becoming an important area for the interactive evaluation of assistant lan- guage models (LMs). Despite their strong performance on static benchmarks (Chang et al., 2025), assistant LMs often fall short of demonstrating their capabilities multi-turn conversations with users (Laban et al., 2025). This gap stems from failure to consider the nuances of user behavior in multi-turn interactions with assistants during evaluation. Humans rarely state their full intent up- front, instead revealing it gradually across turns (Herlihy et al., 2024). Users often provide minimal input because typing requires effort, and sometimes phrase their requests in confusing ways. Evalu- ating assistants under such conditions is critical to understand how they perform when deployed in the real world. This raises the question: how do we simulate realistic human users to replicate their conversational behavior with assistant LMs?
Recent studies that employ user simulators typically rely on prompting assistant LMs to role-play users (Li et al., 2024). However, because these models are post-trained to be perfect "assistants", they tend to produce cooperative and structured user turns, failing to capture the nuanced and often inconsistent behavior of real users. As concrete example, prior work has pointed out that assistant- based user simulators rarely end conversations, choosing instead to chat endlessly (Ivey et al., 2024). Low-quality simulators ative behavior can result in overestimating the performance of assistants. In this study, we address this by introducing User Language Models (User LMs) models post-trained to simulate users that can interact with assistants. A crucial contribution of our work is to train user LMs that can be conditioned on a user intent, enabling us to steer conversations towards tasks we want to study (e.g., solving coding or math problems) while replicating human conversational behavior.
in turn problematic for evaluating assistants, since their overly cooper-
Figure contrasts two user simulators intending to solve a coding problem: a prompted assistant model (GPT-44) and our user language model (Userl.M-8h). The GPT-4o-based simulator produces user turns that are straightforward and direct, making it relatively easy for the assistant to complete the task. In contrast, our UserLM-8h generates user turns that more closely resemble the indirect ways real users phrase requests. Although the core task content is still present, these nuances detract the assistant and ultimately lead the assistant to fail at solving this task.
We first describe the details that are necessary to efficiently train user LMs ($2), which we demon- strate by measuring distributional alignment (i.e., perplexity) with real human utterances on out-of- domain data ($2.3). We then present a set of six evaluations that focus on assessing fine- ane-grained properties of user LMs, specifically around multi-turn interaction and simulation robustness ($3). Our results show that user LMs achieve superior performance at generating more diverse user turns, decomposing intent across turns, and terminating dialogues ($3.3) compared to existing methods. We further show that user LMs are more robust simulators, adhering better to their user role and intent compared assistant- simulators. Finally, we show that deploying the user LM sim- ulate coding and math conversations with GPT-4o assistant leads to more realistic estimate of real-world performance on these tasks than using prompted assistants as user simulators ($4). We end the paper with a discussion and scoping of our findings ($5), and release our trained user LMs publicly to stir further research user simulation, which we hope to go hand-in-hand with the development of robust assistant LMs.
TRAINING USER LANGUAGE MODELS
2.1 PROBLEM DESCRIPTION
Our objective to train a user LM that mimies human behavior when interacting with assistant LMs and performs three key functions: (1) initiate conversation with the assistant given a defined user intent, (2) follow-up with the assistant based on its responses in subsequent turns, and (3) end the conversation once it has run its course. To achieve this, our approach leverages real human-assistant conversations as training data and "flips the dialogue" to train the UserLM to model the conditional distribution of user utterances: at the first turn conditioned turns conditioned on both the intent and the state of conversation so far. This setup is illustrated in Figure 2, which now describe in more detail.
user intent, and at subsequent
2

---

Published as a conference paper at ICLR 2026
User-Assistant Conversations
User Intent Generation
User Language Modeling
th-thewil usarassicint conversctions
Genwatte geowio user intette comversatione
FUD the daloouee creade trkiingg samples User
DON VERSAT ON 1
locnaraaiinn
SAMPLE
use esstegie guageemmoo mocomo
does Bleriine cause weight gain
yeatss samplex
Tum
al Manguugee rtodkt soset personal eupertrcesa dit sccondingto.. to...
X
INTENT 1
OUTPUT:
doee allertine weightt gais
do anthissamines cause weight gain
You'as talking witt aocdetent mrdegemmmee get infermation about stratogles for weight and the impast cartain drugs or medicationa on weight gen
You are a cse cheating with ansittant larguage
mood to get mometio about stseeeee for losinng...
kostt anshorariies do gain as ssee affect Howawer
SAMPLE
does altertine cause weight gain
Tum 2
how wwwhhweight can lose outting sugar for 40 days
expariszee Bor eccording to
you
X INTENT N-1
OUTPUT: anth anterintee cousse veiigtt gain
CONVERSATION N-1
Youare a uter tiiking with ussaiita aogiaaattaagee wwaggggggmmooooo to.
SAMPLE Tum inseraction
CONVERSATION N
G INTENT N
Is einstain determiniin challenged by mecharize?
cooe lokkng with assintant
understand model
SAMPLE Tum K Conversatio Ending
"ootel meiinfomation stronegie looing
how much weight can lese by outting sugar days
Coars mechanics challenges Ciratacss
Mechanics parspeccion
gel
say?
The
looe...
It ocotts you are rofering to Stothes
OUTPUT:
Figure 2: A diagram illustrating our approach to train a UserLM (o+). We leverage in-the-wild coman-assistant conversations, generating high-level user intents for each conversation. We then flip the dialogue, turning each conversation with K turns into K+1 training samples, conditioning both on the high-level intent and conversation state to generate the next user utterance.
Defining User Intents. Similar to how assistant LMs must follow instructions, user LMs must follow an intent that directs the conversation. Initial experimentation revealed that there exists a fine balance in defining user intent. On one extreme, not providing any intent at all reduces the usability of the simulator, as it cannot be effectively directed towards task. On the other extreme, fully- specified intent that contains all information of the user utterances renders the simulator obsolete: simply parrots information from the intent with little added value. In this work, we define user in- tents as high-level conversation objectives: capturing the overall goal of the user without mentioning explicit details (see examples in Figure 2) achieving balance between the two extremes. Prelim- inary analyses showed that user LMs trained on high-level intents were practically more useful than ones trained without intent or with fully-specifin intents: they can be guided to simulate specific conversations, but are in charge of language choices that allow for diverse and realistic simulation.
Ending the Conversation. Humans naturally end conversations a signal that often indicates the conversation has run its course (i.e., the user has obtained the information they seek, completed a task, etc.). Typically, users disengage without providing explicit feedback to the assistant. To replicate this behavior, it is essential that the user LM can effectively decide when to end the conver- sation. We implement this by adding special </enconee sation|> |> token to the tokenizer, which we then use as the output to generate after the last assistant turn in each conversation.
2.2 EXPERIMENTAL DETAILS
Training Data. We use WildChat (Zhao et al., 2024), which contains 478,498 English conversa- tions between ChatGPT and users from 192 countries. We noticed many near-duplicate samples and performed dc-duplication, which resulted in 384,336 conversations (sce details in Appendix A). The authors of WildChat did not define data splits (i.e., train vs. test); therefore, we created our own by first identifying unique users based their hashed IP addresses and countries. For each country, randomly split users into a 90/5/5 split for training, validation, and testing. All conversations from the same user were then assigned to the same split, ensuring that each split contains conversations from entirely distinct users. This resulted in 343,951 conversations for training, 22,442 for valida- tion. and 17,943 for testing. After formatting conversations as we described in $2.1, we obtained 1,047,930 training samples, 1 118,291 validation samples, and 137, 37,224 testing samples.
Intent Generation. For each of the 384.336 conversations in WildChat, we generated generic user intent using few-shot prompting with GPT-4o. The model was given the entire conversation history between the user and the assistant and instructed produce high-level summary of the
3

---

Published as a conference paper at ICLR 2026
user's intent, abstracting away specific details. To guide this process, we provided three manually crafted examples of generic intents as demonstrations (see exact prompt in Appendix D).
Training Details, We perform full- parameetr fine-tuning of Llama3-8b-Ban Base. As we will show, we find that starting from the base model is better than an instruction-tume checkpoint. We used a max sequence length of 2048 tokens, a batch size of 1024 samples, and a learning rate of 2e-5. Training was done on 4 A6000 GPUs, taking 62 hours to train the 1b model and 227 hours for the 8b model. We refer to our fine-tuned user LMs as UserLM-1b and UserLM-8b.
and Llama3.2 3-b--b
Baselines. We compare multiple assistant LMs: L 1-D-nttree-, -bnstreb- and GPT-4o. Prompting assistant LMs to role-play users is the predominant approach in prior work on user simulation (Chang al., 2025; Ivey al., 2024). We further engineered the prompt of the assistant-basee simulators to give explicit instructions on how to replicate user behavior: splitting content across turns, not being verbose. or making occasional typos, without overdoing such phenomena. Starting from prompts used in prior work, we selected the best-performing variant based on validation perplexity on WildChat (see prompts and selection details Appendix D). We also compare to USP-8b (Wang et al., 2025), fine-tuned version of Llama3-8b- 8b-Babe of 94,874 LMSys-Chal conversations to generate user utterances (Zheng et al., 2023). All baselines were instructed to respond with <| sation) if they believe the user intent is fulfilled.
subset
2.3 INITIAL ANALYSES
Distributional Alignment. We evaluate models "user language modeling" how well they match the distribution of human language on the held-out test set of conversations. To measure this, we compute model perplexity (PPL), a standard intrinsic metric commonly used to assess how well an LM predicts a sample of text (Brown et al., 1992; Hewitt et al., 2023). For a test set that consists of a total of N tokens, we compute the per-token perplexity for a model Po as exp (- = log Po IZI))
Lower PPL reflects less surprisal and better alignment with the underlying data distribution. We evaluate on held-out test samples from WildChat and on the entire PRISM dataset (Kirk et al., 2024), which conversations. We applied the identical intent generation procedure to PRISM conversations.
out-of-domain test and consists of 8,011
Results. Table shows the PPI. achieved by base, assistant, and user LMs on the user utterances of the WildChat test set and the PRISM dataset. We com- pare results where models
WildChat
PRISM
Model
A (L)
& (1)
A
2 (
Llama3.2 3amma32211 2-1b-Base
37.68
29.09
84,00
53.54
not conditioned on in- tent at prediction time A ) vs. including the generic intent for conditioning (2)). Three trends are note- worthy. First, intent conditioning leads to gains in PPL for all models across both datasets, confirming that generic intents enable the effective stecring of LMs. Second, though all models have higher PPL on PRISM than WildChat, performance trends on the two datasets are consistent, confirming that PRISM serves more challenging, out-of-domain test set that effectively measures generalization of user lan- guage modeling. Third, the UserLM-88 M-Bb achieves the lowest PPL by a wide margin, often 60-70% lower than all basclines. Finally, the improvements observed when scaling the UserLM from 1b to 8b are encouraging, and we hypothesize that further scaling will likely yield lower PPI. and better distribu- tional alignment. In short, we find that MserLM-8h is more effective than baselines at modeling an out-of oft-oo-domain user population and effectively leveraging generic user intents.
Llamu3-8h-Bann
98.29
48.13
89.98
40.86
Llama3.2 2-Ib-Instruct
26.08
16.08
35.02
20.80
Llama3-8h Sh-nst-t
26.19
21.40
40,25
36.29
USP-8b
32.08
21.78
50.91
30.16
Ms
7.78
18.58
10.33
Use
5.60
4.33
14.92
7.42
Table 1: Perplexity (PPL) of prompted and trained models on WildChat test set (17 (17.943 conversations) and the PRISM dataset (8,011 conversations). We compare PPL with no test-time conditioning (A) and conditioning the generic intent (=).
Do we need to train with intent? The results indicate that even models not trained with intents (such as 1.ma-3.2--l--Bas can effectively leverage them (lower PPL). To gain understanding of whether training with generic intents is truly beneficial, we ablated this parameter by training user LMs without intent conditioning. In this setting, the first user turn is conditioned only the user role tokens (e.g., <|start header id >user<<<enn header id]> in the Llama tokenizer). Figure 3a compares the PPL. of user LMs trained with and without intent for different test-time conditioning
4

---

(b) PPL on PRISM comparing user Published as a conference paper at ICLR 2026
Starting Checkpoint
UserLM-1b
UserLM-8b
22.5
Train Condition ng Inteet
None
22.83
15.96
Train Conditioning Intert
Nove
12.92
8.40
17.5
-12
20
18.45
g
E
15
9.32
15.0
16.33
10-
7.42
12.5
Intent
Inteett
Conditioning
Conditioning
UserLM-8b
(a) PPL on PRISM comparing user LMs trained with with- out generic intent conditioning. Models trained with intent more closely match the distribution of user utterances at test time.
LMs trained when starting from base vs.
instruction-tuned checkpoints.
Figure 3: Comparison of different training setups for our user LMs: (a) Effect of training with con- ditioning the generic intent; (b) Effect of starting from the base instruction-tunee checkpoints.
setups. The models trained without any intent conditioning still show improvements when condi- tioned on intent at test time. However, we find the biggest drops in PPL with models trained with intent. In summary, the use of cedeettconditionnng at train-time leads to improved sensitivity to intent trained user LMs, resulting more steerable and usable model.
Is starting from instruction- fundd checkpoints better? We investigate whether training UserLM by starting from an instruction-tuncd checkpoint leads to better performance than from a base model. Figure 3b compares the PPL. achieved by our 1b and 8b user models when trained from either checkpoint. We find that user models trained from the base checkpoint achieve better results with lower PPL. Instruction-tuned models are typically trained be helpful assistants using synthetically generated data (Gan & Liu, 2024), which we hypothesize semantically distant from user behavior, leading lower distributional fit. On the other hand, base models trained natu- ral text, a majority of which could numan-writeen and is thus initially closer to the distribution of real users. This experiment and additional evaluations reported in Appendix B.2 provide evidence for a high-level observation: base pre-trained LMs are neutral, general-purpon models that can be post-trained towards distinct and opposing roles: user and assistant LMs.
EVALUATING ALIGNMENT WITH HUMAN BEHAVIOR
Beyond distributional measures such as perplexity, we introduce more fine-grained evaluations that capture key properties user simulators should reflect if they align with human behavior. We first propose evaluations targeting multi-turn interactions between a user and an assistant, spanning the opening, intermediate, and closing turns of conversation ($3.1). In addition, present evaluations designed to assess the robustness of the simulator in maintaining a realistic user behavior ($3.2).
3.1 MULTI-TURN INTERACTION
T First Turn Diversity. At the first turn, real users express the same request in many ways. To ensure a simulator reflects this natural variability, it important to measure how diverse its first-turn generations are. We measure diversity by assessing how often models generate new words instead of repeating previously used ones. For each model, we randomly sample 2,000 first-turn utterances and compute the pairwise 1-gram Jaccard index, common metric for utterance diversity (Stasaski & Hearst, 2022). A higher value indicates better lexical diversity in first-turn utterances.
dh Intent Decomposition. Human users often omit details in their prompts, phrase requests in varied ways, and rely on the broader conversational context, expecting the assistant to interpret and leverage this information across turns (Herlihy et al., 2024; Yang et al., 2025). It is therefore essential for our user LM to gradually progress through its intent across multiple turns, rather than revealing all the details in single turn. We measure this by computing the average overlap of 1- grams between the generated user turns and the generic intent for each conversation in PRISM. We remove all stopwords before computing overlap. Since the task intents generic language that
5

---

Published as a conference paper at ICLR 2026
does not match the language of real user utterances (as confirmed by the very low n-gram overlap between test intents and real user utterances), a lower overlap indicates that the model rephrases intent information using its own varied language rather than copying verbatim from the intent, while introducing details progressively across turns.
Dialogue Termination. We assess the ability of each simulator to terminate the conversation when the conversation ended with the real user. To measure this, we compare the model's gener- |> token with actual conversation endings in the PRISM dataset. Treating this as a binary classification task, we compute the F1 score quantify how accurately the model predicts conversation endings. A high F1 score indicates that the model not only ends conversations at appropriate points but also avoids stopping too early or going on excessively.
ation of the < <|endconversatin
3.2 STMUILATION ROBUSTNESS
Naturalness. Text generated by a user LM should resemble the natural way humans write and be distinguishable from the typical style of assistant LMs. To measure the naturalness of the utterances, we use the state-of-the-art Al-detector Pangram (Emi & Spero, 2024), which was shown by Russell al. (2025) to achieve near- perr-eerect detection accuracy, on par with frequent users of ChatGPT. For our evaluation, we sample 2,000 first-turn generations, ensuring each utterance ranges between 50 and 200 tokens, enough text for the detector perform its prediction reliably. Pangram returns a likelihood of being human-written, which we average across all 2,000 utterances.
O User Role Adherence. A robust user simulator must consistently maintain its user role and avoid behaving like the assistant. To measure user role adherence, we test models under the fol- lowing setup. First, we initiate conversations where the user asks an MCQ question and provides set of answer choices (e.g.. What's 's the nickname of the monster the beauty loved? The choices are: beast. ugly, ugliness, satellite). We then prompt GPT-4o to generate a response stating uncertainty and seeking help from the user (e.g., I' I'm honestly not sure about that. It sounds familiar, but I can't t confidently say what the nickname is. If if's's from a specific story or version. Could you just tell me the answer instead?). We then generate the next user turn and measure the rate at which models adhere to their user role by avoiding revealing the answer to the assistant. An attempt to answer is counted when a model includes one or two of the choices in its second turn, ignoring cases where all options are mentioned, which we found to be repetitions of the question. We perform this test using 2,000 random samples from the Commonsense dataset (Talmor et al., 2018) as the first user turn.
f Intent Adherence. to clarify, they suggest something the user did not want. Robust user simulators must adhere to their intent and not comply with assistant suggestions that steer away from their intent. To measure this, we initiate conversations where the user asks an open-ended question (e.g., who sang the song i' I'm a nut?). We then prompt GPT-4o to generate a response stating uncertainty of the answer and suggesting to assist with something else (e.g., I'm not sure what the answer is who sang the song "I'm a Nut." However, if you's 'e interested in exploring music, how about looking into the fascinating world of unusual instruments?). We then generate the next user turn and measure the rate which the simulator refuses the assistant's suggestion and adhere to their original intent. We perform this test using 2,000 random samples from NaturalQuestions (Kwiatkowski et al., 2019) as the first user turn. We evaluate whether the model adheres to its original intent by prompting GPT-4o as judge.
Assistants sometimes get confused in a conversation and, as an attempt
3.3 RESULTS& & ANALYSES
Table 2 summarizes the results achieved by all models. We make the following key observations:
User LMs better align with humans multi-turn interaction across the board. User LMs generate more diverse first turns, with UserLM-8B achieving 94.55% unique I-grams, on- on-par with real users (94.01% 1)), and outperforming GPT-4o's 's 74.42%. User LMs are also better at decomposing intent across turns and produce more abstractive utterances, with an average overlap of 2.69% with the conditioned intent close to the 1.68% observed in human utterances. By spreading information across turns and phrasing requests in varied ways, user LMs more closely reflect the dynamics of real human interactions. Further, user LMs are better at recognizing when to terminate a conversation, achieving Fl score of 63.54. In contrast, prompted assistants rarely end conversations, with Fl scores ranging from 3-15. In other words, the assistant nature of the LM is still ingrained, leading to lack of ability to end conversations, which is not reflective of the user role being simulated.

---

Published as a conference paper at ICLR 2026
Multi-Turn Interaction
Simulation Robustness
Simulator
(1)
à (4)
(t)
(t)
O (1)
i (t)
Llama3.22 2-1 1---2----nnntuee
I--Insrucct
81.36
15.72
3.47
0.14
77.55
54.95
Llama3-88 (ammm33-bb-nnstuc
81.31
23.95
3.51
0.20
63.25
78.05
GPT-4o-mini
66.10
9.66
15.31
0.04
80.20
88.70
GPT-4o
74.42
7.68
1.38
3.31
38.85
70.95
USP-8b
94.37
6.33
21.31
77.73
98.05
97.55
M-crLM-1b
90.90
3.07
56.83
78.96
91.30
93.55
UserLM-8b
94.55
2.69
63.54
80.21
93.95
94.65
Human (estimate)
94.01
1.68
90.15
Table 2: Results of user simulators based on prompted assistant LMs and trained user LMs on the intrinsic evaluations for: + first-turn diversity, a intent decomposition, < dialogue termination, ø naturalness, O user role adherence, and intent adherence. When possible, we compute metrics of the real human utterances from PRISM serve reference.
User LMs produce more natural-looking utterances. Pangram assigns average 90.2% confi- dence that real user utterances are not Al-generated, confirming its efficacy at assigning high scores to human wwriten utterances. On the opposite end, utterances generated by prompted assistant mod- els were consistently detected as Al Al-generated, with scores ranging 0-3%. User LMs fared better, with scores ranging from 77-81% %, only slightly lower than real user utterances. This sharpens understanding of naturalness in three ways: (1) user and assistant utterances are distinct text distri- butions, with Al-detectors focused on detecting Al-generated assistant text (the predominant type of AI Al-generated text), (2) simply prompting an assistant does not significantly alter its generation distribution, as generated utterances are still easily detected Al-generated, (3) User LMs learn to generate user-like utterances, demonstrated by high naturalness scores.
User LMs provide a robust base for simulation. The results for role adherence ( show how the three trained user LMs achieve stellar robustness ranging from 91-98%, indicating they maintain the user role even when a simulated conversation introduces ambiguity. On the other hand, assistant- prompted simulators all show signs of shallow instruction following, reverting to their main assistant role in 20-60% of conversations, over-ruling their prompted instruction to behave as a user.
User LMs stick to their task intent better. The results on intent adherence i paint similar picture role adherence, with user LMs achieving robustness scores of 93-97%. indicating pre- dominant choice to follow original intent and avoid distraction. Prompted assistant LMs, on the other hand, are more easily accepting of diversion, which we hypothesize is related the sycophan- tic nature of assistant LMs (Perez al., 2023; Laban et al., 2023) that favor surface-level sense of pleasing the conversational partner over following the original instruction. This finding exposes an important difference between the user and assistant roles: the user is the pilot of the conversation, and must maintain a certain stubbornness towards their stated intent, whereas the assistant inherently supports the user, and should demonstrate flexibility when the conversation drifts. This disparity is one of the key limitations in using assistant-trained models to simulate users.
How does scaling affect performance? Our experiments involved three families of models at varied scales (Llama, GPT4o, and UserLM), enabling us to observe how scaling affects user simu- lation quality. For prompted assistant models, increased model size does not lead to improvements: Llama3-88 outperforms Llama3-1b lb on only one of six metries, and GPT-4o outperforms GPT-4o- mini on two of the six. On the other hand, UserLM-8b outperforms UserLM-1b all metries, showing that scaling the training of User LMs effectively leads to better user simulators.
4 SIMULATING CONVERSATIONS WITH USER LMS
To gain a more practical understanding of the value of User LMs, now deploy them part of an extrinsic evaluation, which uses the simulator to interact with an assistant for solving tasks. We analyze the performance of assistant LMs in multi-tum conversations with the user simulators. As pointed out by Chang et al. (2025), assistant LMs are typically evaluated single-turn settings, and one of the bottlenecks for multi-turn evaluation is the lack of realistic user simulators.

---

Published as a conference paper at ICLR 2026
4.1 EXPERIMENTAL SETTING
Tasks Setup. We adopt the simulation setting of Laban et al. (2025), which simulates multi-turn conversations for various generation tasks. Specifically, we use 65 task intents that involve users completing math word problem (based GSM8K (Cobbe al., 2021)) writing Python pro- gram (based on HumanEval (Chen et al., 2021)). Both tasks are appealing as they represent common use cases of current LLMs (Tomlinson et al., 2025), and can be evaluated in verifiable manner. We use UserLM-8b to simulate 10 conversations for each task intent (a total of 650 simulations). We also simulate conversations by prompting GPT-4o-mini and GPT-4o as the user simulators. In all conversations, we use GPT-4o as the assistant LM, keeping this aspect of the simulation fixed. For UserLM-8b, we applied set of simple generation guardrails (detailed in Appendix C.I) to coun- teract noise from its smaller model size; these guardrails were only used in this section and do not affect the evaluations in Tables and 2.
Evaluation. We performed a quantitative analysis of the simulations, computing eight metrics focused on five aspects of the simulator: (1) Intent Coverage: whether the simulator reveals the intent information adequately in the course of the conversation, (2) Information Diversity: how the simulator repeats, skips, or adds information during conversations, (3) Pace Diversity: whether the simulator can reveal information at varied speeds, effectively simulating users with differing com- munication styles, (4) Lexical Diversity: the capacity of the simulator using unique wording in each simulation, and (5) Assistant Performance: the impact of the simulator on the assistant's 's abil- ity to complete the task. The implementation details of each metric are provided in Appendix C.3.
4.2 SIMULATION RESULTS& & ANALYSIS
User Simulator
4o-mini
GPT-4o
Usc
Table 3 summarizes the results from the simu- Metric lations. We observe the following:
Intent Coverage
Intent Coverage (%)
86.6
84.7
76.7
User LMs selectively reveal and revise their intent while avoiding unnecessary details. The three simulators cover 76-86% of the units of information from the original intent, indi- cating they successfully stay on topic. Our UserL.M-8b is more likely to repeat required information across turns, and less likely to re- veal information that is not required to accom- plish the task. In contrast, the GPT-based sim- ulators more monotonic: they tend to re- veal information from the intent once, with less emphasis revising what they mean by rep- etition or omitting information. UserL.M-8h is the only simulator that introduces additional de- mands not specified in the original intent. We manually labeled the additional demands spec- ified the UserLM-8b, with the three main cate- gories being: (1) providing example test cases (34%, e.g., get www.col.osttteooooc vowel (enel("FULL)) ("FULL") should return U) (2) defining naming conventions (21%. e.g.. The function should be named 'rabbit'), and (3) implementation constraints (20%, e.g., Avoid using the built-in min and max functions). In contrast, the GPT-based simulators stick the script, rarely injecting demands, leading to more homogeneous simulations.
Information Diversity
% Repeal Required
31.8
9.4
54.3
Skkpp Non- RequRRquird
10.9
14.6
37.7
%Add Demands
9.5
1.1
43.8
Pace Diversity
Turn Variance
0.9
0.6
2.8
Turn Range
3.7-5.7
4.0-5.4
2.1-6.7
Lexical Diversity
Unigram Difference
0.43
0.40
0.71
Assistant Performance
Assistant Score
73.2
74,6
57.4
Table 3: Summary of results from simulated con- versations with three simulators (prompted assis- tant LMs, and trained user LMs). Each simulator is evaluated on its coverage of the intent, informa- tion selection, conversational pace, lexical diver- sity, and the average downstream assistant perfor- mance. See Appendix C.4 for per-task results.
User LMs exhibits more turn variance than the GPT-based simulators. Our UserLM-8h dy- namically decides the granularity of information revealed at each turn, leading to more variation with conversations ranging from 2.1 to 6.7 turns. The GPT-based simulators more consistent, with a narrower range of 3.7 to 5.7 turns. This variability is important because it allows M-8b to simulate users with different interaction paces, capturing a broader spectrum of dialogue behaviors.
User LMs simulate more lexically diverse conversations. UserLM-8b achieves higher lexical diversity. In other words, two simulations based on the same intent have less lexical overlap when using UserL. M-8b compared to using GPT-40 as the simulator. The diversity is due to UserLMM-Bb
8

---

Published as a conference paper at ICLR 2026
varying its language and style in different conversations, in contrast to GPT-based simulators that retain high similarity with the intent and avoid language variation.
Assistants struggle more when conversing with User LMs. We find that assistant task perfor- mance is around 17% lower in conversations with UserLM-8b than with the GPT-bbased simulators. As identified above. UserLM-8b exhibits more diverse behavior in the information it presents, in the pace it sets for the conversation, and in the lexical choices it makes during simulation. In turn, these more realistic simulation conditions are challenging to the GPT-4o assistant, offering a more comprehensive estimation of assistant performance in multi-turn interaction with diverse users.
DISCUSSION & IMPLICATIONS
Stronger assistants LMs are not necessarily better user simulators. A repeating insight from our results is that scaling up assistant LMs does not automatically result in improved simulators. We first see this in $2.3 when comparing PPL on user utterances, finding that (lama3--b-Instren ex- hibited worse PPL than the smaller 1b model. Similarly, in our analysis initializing training from instruction-tume checkpoints, starting from the lb assistant model resulted in lower PPL (18.45) compared to starting from the 8b assistant model (27.25). This pattern also emerged in some of the intrinsic evaluations where GPT-4o-mini surpassed GPT-4o, particularly in terms of intent and role adherence. These results highlight the need explicitly train user LMs.
From User LMs to Personalized User LMs. In this work, user LMs were trained to simulate broad, general audience, capturing behaviors commonly shared across a population of users. How- ever, users with different demographics or personas may exhibit important variations that are impor- tant to model. For instance, non-native English speakers in Middle Eastern countries often phrase requests differently from native speakers in the United States, reflecting their own dialects and lin- guistic backgrounds. We believe key focus for future studies is the development of more personal- ized simulators that can simulate specific user groups or behavior in specific domains. The collection of large-scale data in specific settings is often challenging. and we hope the user I.Ms we propose can serve as a foundation to finetune more personalized user LMs, lowering data requirements.
User LMs will serve purposes beyond interactive evaluation of assistants. Beyond interactive evaluation, we believe user L.S can open up range of promising applications. One direction is user modeling, where prior work has explored the use of LMs to estimate distributions of user responses to surveys (Hu & Collier, 2024; Suh et al., 2025). User LMs can be seen generative extension of this approach: instead of predicting distributions, they can generate natural text responses to survey questions, providing estimates of user behaviors. A second use case is improving judge models. Current approaches often train judge models based on assistant LMS to approximate user preferences, but these are prone to assistant-specific biases and limited personalization (Dong al., 2024). By contrast, user LMs could serve as more realistic judges through prompting or finetuning into user LM judge models. Finally, user LMs can be leveraged for synthetic data generation (Wang et al., 2025). As we show, user LMs produce more varied simulations, making them better-suited for generating synthetic data that can be used to fine-tune assistants and improve their robustness.
Simulating Users vs. Working with Real Users. The experiments we present in this work use automated methods to simulate users, presenting findings that are expected to generalize to real user behavior. However, simulating user behavior is a challenging problem, and we do not anticipate simulations to be applicable to all cases. In particular, working with experts in technical domains such as law (Das et al., 2025), creative writing (Chakrabarty et al., 2025) or science (Asai et al., 2024) remains necessary. We propose the following scope: user simulation is useful tool in scaling experiments, allowing the efficient discovery of system flaws that affect broad populations, whereas working with experts specific user populations is necessary to gain knowledge about user nuances.
Building stronger user LMs will help build stronger assistant LMs. The user LMs we release as part of this work are relatively small (Ib and 8b parameters) and have been post-trained on a limited set of natural user-assistant interactions (343,951 conversations). Despite this, they still outperform prompted assistants such as GPT-4o in simulating users and provide a more compre- hensive evaluation of multi-turn assistant performance downstream tasks. We expect that scaling

---

Published as a conference paper at ICLR 2026
up model size and training data will result in better user LMs, serving as more realistic simulation environments. This, in turn, will help develop more robust and effective assistants.
RELATED WORK
Historically, one of the main uses of computers has been simulate environments across many domains from macro-economics (Holland & Miller, 1991; Zheng et al., 2022). climate (Taylor et al., 2012), elections (Straffin Jr. 1980; Le Pennec & Pons. 2023), molecular biology (Shaw et al., 2010), financial markets (Cont & Bouchaud, 2000), self-driving cars (Doso- vitskiy et al., 2017), and robotics (Todorov et al., 2012). Simulation environments are invaluable in these applications as they enable the evaluation of theories or methods, on-the-fly decision-making for a system based on simulation, or the training of AI systems, for instance through reinforcement learning. As conversational interfaces see widespread adoption in society, we view user LMs as foundational simulation environment for user-Al conversation.
Simulation Environments.
Simulating Human Users In Conversation. User simulation has received considerable attention in past research, serving as an environment for dialogue policies to interact with in reinforcement learning setups (Shi et al., 2019). Early conversation simulators were introduced in tase-triented dialogue research, consisting of rule-based systems that define user goal and an agenda of pending actions that dictate the next user move (Schatzmann et al., 2007; Li et al., 2016). These systems later evolved to include neural- based components that encode the dialogue state and generate user actions (Asri et al., 2016; Kreyssig et al., 2018; Gür et al., 2018; Tseng et al., 2021). More recently, pre- trained assistant LMs have been adopted in user simulation research. The majority of existing works have prompted assistant LMS in a zero-shot manner to role-play user (i.e., "You are a user chatting with Al assistant ..."). instructing them to generate the next user turn given the dialogue history (Li et al., 2024; Luo et al., 2024; Zhong et al., 2025; Chang et al., 2025). However, a growing body of work highlights that these prompted assistants exhibit low alignment with real human interactions and often diverge from human behavior (Ivey et al., 2024; Yoon et al., 2024; Zhu et al., 2024).
A few studies have proposed fine-tuning approaches to generate user turns given the dialogue context (Liu et al., 2022; 2023; Sekulić et al., 2024) by training on task-oriented dialogue datasets such as MultiWOZ. (Eric et al., 2019; Cheng et al., 2022). Such fine-tuning approaches have been mainly focused on generating synthetic user-assistant conversations that can be used as additional training data for assistant LMs (Wan et al., 2022; Ferreira et al., 2024; Dhole, 2024), with some focusing on generating diverse user profiles (Kong et al., 2024; Wang et al., 2025). Our work complements this body of literature by introducing general-purpose user language models trained to follow a defined user intent and simulate users in conversations with assistants. We show that user LMs better align with real human behavior compared to prompting assistant LMs to role-play users.
Evaluating User Simulators. There is little work on how to evaluate simulators at replicating hu- man behavior in their conversations with assistants (Deriu et al., 2021). Some studies measure the correlation between simulator and human utterances using lexical or stylistic metrics (Ivey et al., 2024: Pietquin & Hastie, 2013), or the diversity of their generated utterances (Wang et al., 2025). Others have resorted to human annotation to judge the quality of the simulated conversations (Kranti al., 2025; Sekulić al., 2024; 2022; Shi et al., 2019). Another line of work evaluates the simulator more indirectly by testing whether task performance with the assistant correlates with performance in real user interactions (Chang et al., 2025). Our study introduces fine- fine-grained evaluations of simu- lator properties related to multi-turn interaction behavior and robustness. We also assess simulators when deployed for solving coding and math tasks with assistants.
CONCLUSION
In this work, we introduced approaches to train and evaluate User LMs. Through many experiments, we showed the inefficacy of prompting assistant LMs to simulate users and highlighted the value of purpose-built user LMs, trained on user utterances from large corpus of user-assistant interactions. We demonstrated a practical application of user LMs using them to simulate conversations with an assistant. This surfaced limitations in current assistant LMs, which displayed degraded performance due to the more realistic simulation environment. We hope the release of UserLM-88 spurs further research by the community on user language modeling.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We would like to thank Chan Young Park, Jillian Fisher, and Serina Chang, and the Al Interaction and Learning team members Microsoft Research for valuable discussions. We also thank Pangram for providing credits to access their API, facilitating some of our robustness evaluations. Tarek Naous and Wei Xu are partially supported by the NSF CAREER Award IIS- IIS-2144493. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
ETHICS STATEMENT
The Use we contribute in this work is trained from base checkpoint (Llama3-8b-base) which has likely not undergone the conventional post-training mechanisms for output safety and alignment. Thus, model is expected to bchave similarly to a base model, which could be more prone to generating toxic language than an instruction-tus trained generate user utterances that realistic, which also demonstrated to be not easily detectable by current state-of-the-ar detectors of Al-generated text ($3). The purpose of our model to be used for research purposes simulator for interactive evaluation of assistants and sim- ilar beneficial use cases. We hope our work encourages future research on building detectors that differentiate between utterances generated by user LM and ones written by real humans.
model. We also note that our model
REPRODUCIBILITY STATEMENT
Our trained UserLM-8b model will be made publicly available for research purposes. The model was trained using publicly available data, enabling the community to reproduce and expand on our experiments. We report the decoding configuration used for our UserLM-88 in Appendix C.I and all the prompts we used in Appendix D, which are needed to replicate the results in our experiments.
REFERENCES
Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D' arcy, et al. Openscholar: Synthesizing scientific literature with aetreeell-augmented Ims. arXiv preprint arXiv:2411. 14199, 2024.
Layla El Asri, Jing He, and Kaheer Suleman. A sequenice-oo-sequence model for user simulation in spoken dialogue systems. arXiv preprint arXiv:1607.00070, 2016.
Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, Jennifer C Lai, and Robert L Mercer. An estimate of an upper bound for the entropy of english. Computational Linguisties, 18(1):
31 31-40, 1992.
Tuhin Chakrabarty, Philippe Laban, and Shien-Sheng Wu. Can ai writing be salvaged? mitigat- ing idiosynerasies and improving human-ai alignment in the writing process through edits. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp. 1-33, 2025.
Serina Chang, Ashton Anderson, and Jake M Hofman. ChatBench: From static benchmarks to human-Al evaluation, 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
Qinyuan Cheng, Linyang Li, Guofeng Quan, Feng Gao, Xiaofeng Mou, and Xipeng Qiu. Is Multi- WOZ a solved task? an interactive TOD evaluation framework with user simulator. arXiv preprint arXiv:2210.144 14529, 2022.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10.14668. 2021.

---

Published as a conference paper at ICLR 2026
Rama Cont and Jean-Philipe Bouchaud. Herd behavior and aggregate fluctuations in financial mar- kets. Macroeconomic dynamics, 4(2):170-196 2000.
Debarati Das, Khanh Chi Le, Ritik Sachin Parkar, Karin De Langis, Brendan Madson, Chad M Berryman, Robin M Willis, Daniel H Moses, Brett McDonnell, Daniel Schwarcz, et al. Lawflow: Collecting and simulating lawyers' thought processes. arXiv preprint vrXiv:2504.18%4 2025.
Jan Deriu, Alvaro Rodrigo, Arantxa Otegi, Guillermo Echegoyen, Sophie Rosset, Eneko Agiree, and Mark Cieliebak. Survey on evaluation methods for dialogue systems. Artificial Intelligence Review, 54(1):755-88) 810, 2021.
Kaustubh D Dhole. KAUCUS: Knowledge augmented user simulators for training language model assistants. arXiv preprint arXiv:2401. 2024.
Yijiang River Dong, Tiancheng Hu, and Nigel Collier. Can L.M be personalized judge? arXiv preprint arXiv:2406. /657, 2024.
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pp. 1-16. PMLR, 2017.
Bradley Emi and Max Spero. Technical report on the pangram Al-generated text classifier. arXiv preprint arXiv:2402. 14873, 2024.
Mihail Eric, Rahul Goel, Shachi Paul, Adarsh Kumar, Abhishek Sethi, Peter Ku, Anuj Kumar Goyal, Sanchit Agarwal, Shuyang Gao, and Dilek Hakkani-Tur. MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669 2019.
Rafael Ferreira, David Semedo, and João Magalhães. Multi-trait user simulation with adaptive decoding for conversational task assistants. arXiv preprint arXiv:2410. 12891, 2024.
Zeyu Gan and Yong Liu. Towards theoretical understanding of synthetic data in LLM post- tostntraining: A reverse-bottleneck perspective. arXiv preprint arXiv:2410.0/720, 2024.
Izzeddin Gür, Dilek Hakkani-Tür, Gokhan Tür, and Pararth Shah. User modeling for task oriented dialogues. In 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 900-906. IEEE, 2018.
Christine Herlihy, Jennifer Neville, Tobias Schnabel, and Adith Swaminathan. On overcoming mis- calibrated conversational priors in llm-based chatbots. arXiv preprini arXiv:2406.01663, 2024.
John Hewitt, John Thickstun, Christopher Manning, and Percy Liang. Backpack language models. arXiv preprint TrXiv:2305.16765 2023.
John H Holland and John H Miller. Artificial adaptive agents in economic theory. The American economic review, 81 (2):)3665-3) 1991.
Tiancheng Hu and Nigel Collier. Quantifying the persona effect in Ilm simulations. arXiv preprint arXiv:2402.10811, 2024.
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam,, Junghwan Kim, Bowen Yi, et al. Real or robotic? assessing whether LLMS accurately simulate qualities of human responses in dialogue. arXiv preprint arXiv:2409.08330, 2024.
Hannah Rose Kirk, Alexander Whitefield, Paul Rottger, Andrew M Bean, Katerina Margatina, Rafael Mosquera-Gomae Juan Ciro, Max Bartolo, Adina Williams, He He, al. The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. Advances Neural Information Processing Systems, 37: 17:105233-105344 2024.
Chuyi Kong. Yaxin Fan, Xiang Wan, Feng Jiang, and Benyou Wang. PlatoLM: Teaching LLMs in multi round dialogue via a user simulator. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7841-7863, 2024.
12

---

Published as a conference paper at ICLR 2026
Chalamalasetti Kranti, Sherzod Hakimov, and David Schlangen. clem: todd: A framework for the systematic benchmarking of llm-based lask-oriented dialogue system realisations. arXiv preprint arXiv:2505.00445, 2025.
Florian Kreyssig, lñigo Casanueva, Pawel Budzianowski, and Milica Gasic. Neural user sim- ulation for corpus-based policy optimisation for spoken dialogue systems. arXiv preprint arXiv: /805.0666, 2018.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466. 2019.
Philippe Laban, Lidiya Murakhovs' ka. Caiming Xiong, and Chien Wu. Are you sure? challenging Ilms leads to performance drops in the flipflop experiment. arXiv preprint arXiv:2311.08596 2023.
Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. LLMs get lost in multi-turn conversation. arXiv preprint 2.XXiv2505506/200 2025.
Caroline Le Pennec and Vincent Pons. How do campaigns shape vote choice? multicountty evidence from 62 elections and 56 tv debates. The Quarterly Journal of Economics, 138(():703-767, 2023.
Ruosen Li, Ruochen Li, Barry Wang, and Xinya Du IQA-EVAL: Automatic evaluation of human- model interactive question answering. Advances in Neural Information Processing Systems, 37: 109894-109921, 2024.
Xiujum Li, Zachary C Lipton, Bhuwan Dhingra, Lihong Li, Jianfeng Gao. and Yun-Nung Chen. A user simulator for cos-loompltiion dialogues. arXiv preprint arXiv:1612.05688, 2016.
Hong Liu, Yucheng Cai, Zhijian Ou, Yi Huang, and Junlan Feng. A generative user simulator with GPT-based architecture and goal state tracking for reinforced multi-domain dialog systems. arXiv preprint arXiv:22 / 086962 2022.
Yajiao Liu, Xin Jiang. Yichun Yin, Yasheng Wang, Fei Mi, Qun Liu, Xiang Wan, and Benyou Wang. One cannot stand for everyone! leveraging multiple user simulators train task- dialogue systems. In Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pp. 1 1-21, 2023.
Xiang Luo, Zhiwen Tang, Jin Wang, and Xuejie Zhang. DuetSim: Building user simulator with dual large language models for task-oriented dialogues. arXiv preprint 2rXv::2405.3302 2024.
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, al. Discovering language model behaviors with wrrtt-written evaluations. In Findings of the association for computational lin- guistics: ACL 2023, pp. 13387-13434, 2023.
Olivier Pietquin and Helen Hastie. survey on metrics for the evaluation of user simulations. The knowledge engineering review, 28(1):59-73, 2013.
Jenna Russell, Marzena Karpinska, and Mohit lyyer. People who frequently use ChatGPT for writ- ing tasks accurate and robust detectors of Al Al-generated text. arXiv preprint arXiv:2501.1563: 2025.
Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young, Agenda-based user simulation for bootstrapping a POMDP dialogue system. In Human Language Technologies 2007: The Conference of the North American Chapter the Association for Computational Linguistics; Companion Volume, Short Papers, pp. 149-152, 2007.
Ivan Sekulić, Mohammad Aliannejadi, and Fabio Crestani. Evaluating tional search systems via user simulation. Proceedings of the Fifteenth ACM International Conference Web Search and Data Mining, pp. 888- 896, 2022.
conversa-
13

---

Published as a conference paper at ICLR 2026
Ivan Sekulić, Silvia Terragni. Victor Guimarães, Nghia Khau, Bruna Guedes, Modestas Filipavicius, Andre Ferreira Manso, and Roland Mathis. Reliable LLM-based user simulator for task-oriented dialogue systems. arXiv preprint arXiv:2402.13322 2024.
David Shaw, Paul Maragakis, Kresten Lindorff-Larsen, Stefano Piana, Ron O Dror, Michael Eastwood, Joseph A Bank, John M Jumper, John K Salmon, Yibing Shan, et al. Atomic-level characterization the structural dynamies of proteins. Science, 33066002):341-336, 2010.
Weiyan Shi, Kun Qian, Xuewei Wang, and Zhou Yu. How to build user simulators to train rl-based dialog systems, arXiv preprint arXiv: 01300 2019.
Katherine Stasaski and Marti A Hearst. Semantic diversity in dialogue with natural language infer- ence. arXiv preprint arXiv:2205.. 0e5.00119777 2022.
Katherine Stasaski, Grace Hui Yang, and Marti A Hearst. More diverse dialogue datasets via diversity-informed data collection. In Proceedings of the 58th annual meeting of the associa- tion for computational linguistics, pp. 4958 4968, 2020.
Philip D Straffin Jr. Topics in the Theory of Voting. ERIC, 1980.
Joseph Suh, Erfan Jahanparast, Suhong Moon, Minwoo Kang, and Serina Chang. Language model fine-tuning on scaled survey data for predicting distributions of public opinions. arXiv preprint arXiv:2502.16761, 2025.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Qo A ques- tion answering challenge targeting commonsense knowledge. arXiv preprint arXiv: 1811.0993,, 2018.
Karl E Taylor, Ronald J Stouffer, and Gerald A Meehl. An overview of cmip5 and the experiment design. Bulletin of the American meteorological Sociery, 93(4):485 2012.
Guy Tevet and Jonathan Berant. Evaluating the evaluation of diversity in natural language genera- tion. arXiv preprint arXiv:200.029990, 2020.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference intelligent robots and systems, pp. 5026-5033 IEEE, 2012.
Kiran Tomlinson, Sonia Jaffe, Will Wang, Scott Counts, and Siddharth Suri. Working with ai: Measuring the occupational implications of generative ai. arXiv preprint arXiv:2507.00 07935, 2025.
Bo-Hsiang Tseng, Yinpei Dai, Florian Kreyssig, and Bill Byrne Transferable dialogue systems and user simulators. arXiv preprint 1xxx:22007.11904 2021.
Dazhen Wan, Zheng Zhang. Qi Zhu, Lizi Liao, and Minlie Huang. A unified dialogue user simulator for few-shot data augmentation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3788-3799, 2022.
Kuang Wang, Xianfei Li, Shenghao Yang. Li Zhou, Feng Jiang, and Haizhou Li. Know you first and be you better: Modeling human-like user simulators via implicit profiles. arXiv preprint arXiv:2502.1896 18968, 2025.
Chenyang Yang, Yike Shi, Qianou Ma. Michael Xieyang Liu, Christian Kästner, and Tongshuang Wu. What prompts don't say: Understanding and managing underspecification in LLM prompts. arXiv preprint 2r0iv:2505. 200..55555300 /3360, 2025.
Se-eun Yoon, Zhankui He, Jessica Echterhoff, and Julian McAuley, Evaluating large language mod- els generative user simulators for conversational recommendation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 1490-1504, 2024.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. WildChat: IM ChatGPT interaction logs in the wild. arXiv preprint 07X1422405.01470, 2024.
14

---

Published as a conference paper at ICLR 2026
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P Xing. et al. LMSYS-Chat- Chat-IM: A large-scale weal-world LLM conversation dataset. arXiv preprint 1XXx:2339.11999 2023.
Stephan Zheng, Alexander Trott, Sunil Srinivasa, David Parkes, and Richard Socher. The ai economist: Taxation policy design via two-level deep multiagent reinforcement learning. Science advances, 8( :(18)eebbk660, 2022.
Qishuai Zhong, Zongmin Li, Siqi Fan, and Aixin Sun. Evaluating LLM adaptation to sociodemo- graphic factors: User profile vs. dialogue history. arXiv preprint arXiv:2505.21362. 2025.
Lixi Zhu, Xiaowen Huang, and Jitao Sang. How reliable is your simulator? analysis the limi- tations current Ilm-based user simulators for conversational recommendation. In Companion Proceedings of the ACM Web Conference 2024, pp. 1726- 1732, 2024.
15

---

Published as a conference paper at ICLR 2026
A ADDITIONAL EXPERIMENTAL DETAILS
WildChat Deduplication. During initial analysis, we noticed a lot of near-duplicate conversa- tions in WildChat often consisting of prompt templates with minor input variations. To improve the quality of the training corpus, we applied 7-gram counter on the first-turn user prompts and manually examined the most frequent patterns. We identified several recurring prompt templates, with one used in as many as 81,196 conversations, that do not represent natural user interaction and could degrade model quality. Removing all near-duplicates reduced the dataset to 384.336 conver- sations. To verify the efficacy of our filtering and data splitting, we computed the 7-gram overlap between the first-turn user prompts in the train and test sets, finding only 4.19% overlap. This low overlap confirms that our deduplication and user- based splitting procedure effectively minimizes data leakage between splits.
16

---

Published as a conference paper at ICLR 2026
UserLMM-1 lb
UsertMM-B Bb
Lbma3.21b-instrutt
Liama3-80 B-mn3-----nstucct Instractt
GFT-4o
USP-Bb
Human
Cumulative % 1-gram Overlap
60-
Cureulative 3-gram Overtap %
Curuuaativ Overlap
40
15
Turn
Turn
Turn
Figure 5: Cumulative n-gram overlap between generated user turns and the generic intent of each conversation. Results are averaged across each turn for all conversations in PRISM. Our user LMs achieve the lowest cumulative n- n-gram overlap with the intent, aligning with real human utterances.
UserLM-1b
UserLM-Bb
GPT4o-mini
USP-Eb
1.00
1.00
1.00
Precision
0.75
0.75
0.75
0.50
0.50
0.50
0.25
0.25
0.25
0.00
0.00
0.00
Turn
Turn
Turn
Figure 6: Precision, Recall, and score per turn for our dialogue termination evaluation. Results are shown for models that achieved more than an average Fl score of 10.
B ADDITIONAL RESULTS
B. PER-TURN ANALYSES
Uamma322 2-1b-nnstruc
Perplexity. Figure 4 shows the token-level PPL
Uamas-Bbbiintrruct
70-
achieved by all models on the human utterances of
60
UserLLM-1b
the PRISM dataset. Our user LMs outperform the instruction-tuned models and the USP-8b baselines. Particularly, find the largest PPI. differences be- tween our user models and all baselines to be at the first turn. This indicates that our user LMS are much better at initiating the first turn of a conversation the way real users would, but prompted instruction- tuned baselines struggle to do so.
10
Turn
Figure Per-turn token-level PPL achieved by models on PRISM utterances. All models are conditioned on the generic user intent of each conversation. Our user LMs outperform baselines and achieve much lower PPL, especially at the first turn.
Intent Decomposition. Figure 5 shows the cumu- lative I-gram overlap across turns between the gen- erated user turns and the generic intent we condition Our models have the lowest overlap across turns, aligning closely with humans and outperform- ing the GPT-4o simulator that is explicitly instructed to split intent across turns and not to copy. The instruction-tune Llama models show the highest overlap, starting 25% which indicates higher levels of copying from the intent at the first turn.
Dialogue Termination. Figure 6 shows the precision, recall, and Fl score per-turn for the dia- logue termination evaluation. We find that prompted assistants such as GPT40-mini achieve high precision, yet very low recall, indicating that while their termination usually aligns with real users, they rarely do so and often proceed to continue the conversation. This behavior becomes worse with better assistants such as GPT-4o, which rarely terminates dialogues, achieving near-zero Fl score. Our Us and UserL.M-8h show a better balance of precision and recall, aligning better with human behavior. We note that in practice, decoding can be constrained in LMs to not generate the </endconversation >> token if desired.
17

---

Published as a conference paper at ICLR 2026
Multi-Turn Interaction
Simulation Robustness
Initialization
(t)
ch (U)
+ (1)
(1)
Ø (1)
i (1)
Instruct
89.02
8.40
45.21
53.79
91.13
98.00
Us-r
Base
90.90
3,07
56.83
78.96
91.30
93.55
Instruct
89.04
8.88
56.96
56.54
95.45
97.75
User
Base
94.55
2.69
63.54
80.21
93.95
94.65
Human
94.01
1.68
90.15
Table 4: Additional evaluations on intrinsic properties comparing our User LMs when training from hase checkpoints (Ilama3.2- 2-1-..-- and Llama3-8b-Base instruction-tuned checkpoint (Llama3.2) I-ntruuut and Llama3-8b- 8b-Innurucc)....... The user models trained from a base checkpoint achieve overall better results than ones trained from instruction-tuned checkpoints.
B.2 STARTING CHECKPOINT ADDITIONAL ANALYSIS
Our earlier experiments showed how User LMs trained from a base checkpoint achieve better dis- tributional alignment (lower PPL) human utterances ($2.3), suggesting better simulators. We further verify this by evaluating models trained from either checkpoints on our intrinsic properties presented in $3. Table 4 shows the results, where find that User LMs trained from the base checkpoints generally outperform ones trained from instruction-tune -undd models. Most notably, mod- els trained from the base checkpoints are better at intent decomposition, dialogue termination, and produce text that more closely resembles the style of real humans. Models trained from instruction- tuned checkpoints show similar performance terms of role and intent adherence. These results further confirm the findings from our initial PPL analyses, which show that models already trained be assistants are more difficult to train to become user models.
18

---

Published as a conference paper at ICLR 2026
C CONVERSATION SIMULATION EXPERIMENTAL DETAIL
C.I GENERATION CONFIGURATION FOR USERLM-8B
In the simulation experiment detailed in Section 4, we implemented few simple generation guardrails to counteract the small nature of the model (8b parameters), we found that simply sampling unfiltered responses from the model did not result in satisfactory quality necessary to con- duct simulation experiments.
Guardrail 1: Filtering First Tokens. In our experimental simulation setting, initial trials revealed that the MserLM-8b frequently generates responses that started with "T", "You", or "Here", which led to repetitive user utterances. We constrained the decoding by implementing a logit filter, in which a set of tokens (3 words listed above in lower- and capitalized format) had their logits set to 0 for the first token, ensuring they are not sampled as first words. This simple modification was effective practice at steering the model towards more useful generations.
Avoiding Dialogue Termination. In extrinsic evaluation, we prohibited the model from generating the <|endconversation]) token that we used in our earlier analysis on dia- logue termination capability, by setting its probability of being generated O.
Guardrail 2:
Guardrail 3: Maximal and Minimal Length Threshold. The experiment we conduct specifi- cally evaluates assistant behavior in multi-turn, under-specified conversations. To encourage such behavior from the user simulator, we filtered out any user utterance of more than 25 words, as found through inspection that longer utterances tended to reveal the entire problem formulation, re- ducing the need for multi-turn interaction. In case the UserLM-8b generates response longer than the set threshold, we discarded the response and generated new one, until we obtained response below the threshold. We also filter responses that contain fewer than three words, though note that such occurrences were rare in our inspection.
Guardrail 4: Filter Verbatim Repetitions. Inspection simulated conversations further re- vealed that the Useb can, occasion, repeat previous user utterance verbatim, or copy verbatim the original intent, which does not advance the conversation appropriately. We therefore filter out such responses and re- re-generate using the simulator until a novel response is generated.
The four guardrails detailed above were implemented to maintain a quality in the simulated conver- sations we include in analysis. We believe that these guardrails needed due to the small size of our model, leading to noisy responses. We hypothesize that the need for such guardrails would be alleviated with a larger and more performant user LM, which we hope will arise in the future as the community develops high-quality simulators.
C.2 ADDITIONAL EXAMPLES OF SIMULATED CONVERSATIONS
The simulation experiments Section conducted ten simulations for each task instruction, allow- ing us to observe the diversity of simulations that can occur. For given task, Figure 7 provides three simulated conversations using GPT-4o for the simulation, and three conversations simulated using UscrLM-8b. The UserLM--b-bassel conversations more unique terminology (e.g., "ignore" vs. "keep" or "translate" vs. "replace"), which we measure quantitatively by observing higher pace and lexical diversity. This additional diversity in simu- lation leads to more challenging environment for the assistant, which is measured to have lower performance than when conversations are simulated using GPT-4o.
more distinct from each other, with
C.3 DEFINITION OF SIMULATION EVALUATION METRICS
We now define the eight metrics used as part of analysis in Section 4 that involves the simulation of multi-turn conversations using various user simulators.
Each simulated conversation in the experiment is based the assistant from the Lost in Conversation corpus (Laban et al., 2025), which we use as the user intent of the simulation (i.e., "You are user chatting with assistant language model complete
instruction that the user provides to
19

---

Published as a conference paper at ICLR 2026
SIMULATING USERS IN CONVERSATIONS
USING ASSISTANT LANGUAGE
USING LANGUAGE
Intent: Python given errey integers, inclusive, reverse the arryy. and replace digits by their name from "One", "Two", "Three", etc.
GPT-40
Start with list numbers.
having numbers.
write a python
UserLM-8b
Turn digits into list
function that accepts an of integers
the ints between and 9 using the merge algorthm..
Ignore any number
that is not the
Only numbers from to 9, sort them needed.
step sort numbers they're between 1 and
1-to-9 range
koop only the integers between and 9
the numbers they'r re between and 9
now add sorting into
Now! want to flip the list around after sarting
Next, we can flip the list around after sorting
sort those kept integers
want flip the liet after that
Replace each digit by Its corresponding name from "One" "Tvi", 'Throa", 'Four,. "Flve", "Sx", "Seven", "Eight", "Nine".
now reverse the sorted list
Reverse that sorted
wwnn to use names for numbers from
words for numbers, like One Nine.
wards the numbers from One Nine.
translate each remaining digit into capitalize English name
translate each remaining digit into its English name, from Onn' through 'Nine'
Figure 7: Selected simulated conversations for fixed instruction, using a prompted GPT-4o (three simulations on the left) and using UserLM-8b (three simulations on the right). We only show user turns, and omit assistant responses. UserLM-baa M-based simulations are more diverse, using more varied language in each, as confirmed quantitatively bu our vetrics-based analysis.
the following: [INSTRUCTION[), Each instruction is declined as a fully-specified instruction: single utterance that defines the instruction in its entirety, and a sharded instruction: a set of 3-8 "shards" that taken together, reveal the same information the fully-specified instruction.
For our analysis, we consider shards as atomic units of information from the original fully-specified instruction, which enables to scrutinize which information is and isn't revealed by user simula- tor. Given a simulated conversation based on an instruction, we prompt OpenAl's 's o3 model to assign which specific shard (or shards) revealed in each user Manual inspection of 5 simulated conversations revealed that o3's mapping ability was accurate to conduct our analysis. The Intent Coverage metric is then defined
len (en(sse(revealeddshart))))) # shards
Intent Coverage
in other words, intent coverage measures the fraction of shards that were revealed at least once during simulation.
Sharded instructions have additional metadata attached to each shard, namely whether the shard is required or not to solve the task. This Boolean tag enables us to analyze which parts of instruction they focus on. First, we calculate:
Repeat Required = {i
all required shards are revealed at most once, otherwise (i.e., at least one required shard is repeated 2+ times)
(2)
In other words, this measures the percentage of conversations in which the user simulator highlighted important information by repeating across turns.
We also compute:
Skip Non-Required 0 all the non-required shards are revealed,
(3)
otherwise (i.c., at least one non-required shard is ommitted)
which measures whether the user simulator is selectively picking secondary information to skip over during simulation. Non-required shards are typically obvious information (such as a trivial base case clarifying example).
20

---

Published as a conference paper at ICLR 2026
Finally, used o3 to identify any information introduced by the simulator in the utterances that was not initially present in any of the shards. This information is then used to compute:
{ if no novel information is in the simulated conversation, otherwise (i.e., 1+ user utterance introduces novel information)
Additional Demands
In Section 4, we perform qualitative analysis of the additional demands introduced by user simula- tors, which are categorized into three main types: providing examples, defining naming conventions for functions or variables, and adding implementation or algorithmic constraints.
The above-defined metrics all computed for each individual simulated conversation and then averaged across each user simulator. In Table 3, each result therefore represents the aggregation of 650 simulations (65 instructions, and 10 simulations per instruction).
The next metrics we define require multiple simulations for a given instruction: given that the simu- lators probabilistic in nature (i.e., language models run at temperature T 1.0), each simulation diverges and is unique. We simulate a total of 10 conversations (C1, C2,..- C1o) for each instruction for each simulator, and compute the following metries based on the set of simulations obtained for each instruction:
Turn Variance = varr21 (#turns(Ci))
(5)
where war(X) X') is the empirical estimate variance of the series, and #turns the number con- versational turns (user-assistant exchanges) in the simulated conversation. We also report the range of the number of turns, to help elucidate whether an increase in variance comes from the lower or upper range of conversation turns. The range is computed as:
Turn Range [min((#urnss(C) max tur(Hturss())))
(6)
for each instruction. We compute lexical diversity following prior work (Stasaski al., 2020; Tevet & Berant, 2020) by calculating pairwise-lexier overlap of unigrams:
lintersection(C: GiM1
Unigram Overlap
10C2
lunion((C, Cj)l
In more specific detail, each pair of simulated conversations on a common instruction, we ex- tract the lemmatized unigrams of all user utterances and measure the Jaccard index. We average this across all pairs of simulated conversations, effectively measuring the average overlap in wording across two simulated conversations based the same instruction. We then calculate the comple- ment:
Unigram Difference = Unigram Overlap,
(8)
which assesses the simulator's ability use unique wording in each conducted simulation.
Turn variance, turn range, and lexical diversity are computed for each instruction (based on 10 simulations), and aggregated across instructions.
As final metric, we assess whether the assistant successfully completes the task (i.e., generates Python code that solves held-out unit tests or finds the correct mathematical solution to the problem), which is computed as:
f if an assistant solution at any turn passes evaluation, 0 otherwise (i.c., none of the attempted solutions correct).
Assistant Score
(9)
This final metric does not directly evaluate the user simulator, but instead measures the effect of the simulator on the downstream performance of the assistant (GPT-4o), giving us insight into the impact that the user simulator has the task-solving ability of the assistant.
21

---

Published as a conference paper at ICLR 2026
Task: Math
Task: Code
4o-mini
GPT-4o
UserL
4o-mini
GPT-4o
UserI. 8h
Metric
Intent Coverage
Intent Coverage (%)
95.1
76.5
82.8
76.6
Information Diversity
Reppaat Required
14.5
44.0
3.0
39.6
$8.9
&Skip Non-Required %Add Demands
10.9
14.6
37.7
10.0
0.5
40.2
9.3
1.1
45.5
Pace Diversity
Turn Variance
0.4
3.4
1.2
2.6
Turn Range
4.0-5.3
5.3-5.4
2.0-6.8
3.5-5.9
3.5-5.4
2.2-6.7
Lexical Diversity
Unigrum Difference
0.32
0.30
0.68
0.47
0.43
0.72
Assistant Performance
Assistant Score
0.83
0.85
0.46
0.69
0.70
0.63
Table 5: Summary of evaluation metrics for simulated conversations across two tasks types (math and code) with three user simulators (prompted assistant LMs and trained user LMs). Each simulator is evaluated on intent coverage, information diversity, conversational pace, lexical diversity, and downstream assistant performance.
C.4 SIMULATION RESULTS PER-TASK
Our main results presented in Section aggregate metric scores across the coding and math solving tasks to give sense of the simulator behavior in general. We show the results for each of those tasks in Table 5. We note that for the math instructions, all shards were required, rendering the calculation of Skip Non-Requi red not possible.
Overall, we observe similar trends across the two tasks. Some noteworthy differences: on math tasks, the GPT-based user simulators stick the script more strictly, with almost 100% coverage of underlying shards, almost no turn variance, and a higher gap in observed assistant performance with the UserLM-8h On the other hand, all simulators exhibit little more diversity in their responses in the coding tasks, with better lexical and turn diversity.
22

---

Published as a conference paper at ICLR 2026
D PROMPTS
Intent Generation for Conversations. Figure shows the prompt we used to generate a generic user intent for each conversation in Wildchat and PRISM datasets. We provide GPT-4o with the full conversation between the user and the assistant and instruct it to create a short summary of the high- level intent of the user from the conversation, without mentioning any specific details. We provide 3-shot examples where we manually wrote the generic user intent from the conversation.
User Simulation with Assistant LMs. Figure shows the prompt we used to simulate users with assistant LMs, which we based on how recent studies have done it (Chang et al., 2025; Ivey et al., 2024). The assistant given the generic intent of the user and instructed to generate the user utterance at the first turn and subsequent turns given the conversation history so far. We further engi- neered the prompt to provide additional instructions on the desired behavior of user utterances (i.e.. intent decomposition, making occasional typos, writing too verbose utterances, etc.), without overdoing these phenomena.
Ludge-Judge for Intent Adherence. Figure 10 shows the prompt we used classify if the user simulator successfully adhered to its intent in our intent adherence evaluation. We provide GPT-4o with the user simulator question to assistant, the assistant's refusal to provide the answer and suggestion of looking something else, and the user simulator's reply. Based on this conversation history, we instruct GPT-4o to classify if the user simulator stuck to its intent by reiterating what they asked in their first turn, whether they complied with he suggestion of the assistant.
23

---

Published as a conference paper at ICLR 2026
Prompt Template for Intent Generation
You are given the conversation history between user and assistant model and your task is to create sumsary of the user's s intent from the conversation.
stractured define high Level intenc is, should not go into specific details..
summary
Fornat the start with "You chatting with assistant language model
Exarple 1:
Conversation History:
suser:: List to me besutiful sights in Algeria
<assistant>: Algeria breathtaking sights you Tassils nt Agjer Natsonal Park,, tjemila, Tingad, Band Hamma Fort, Basilique Notre Dane d'Afrique
of striking natural and rich cultural heritage.
Intent Sumary::
You are user chatcing with assistant language model to obtain some recomendations places see In Algorta.
Example 2:
Conversation History:
Suser>: does
cause weight gain
<assistant>: As an language rodel, don't have personal experiencas. But according the available literature, Allertire
Suser?: do ant iistaiissss cause wright gain
<assistant>: Host antihistantres
effect. However, people Find that they expar increased appetite cravings sugary, higircalorie
Quser>:: hox nuch weight can Tose by cutting for
cassistant>: anount weight you lose by cutting sugar your dist days depand on a variety of factors, including your starting wigght age, gender, level of physical activity,
Intent
You are usar chatcing with assistant language model to get information about stratogles For Lesing weight and the impact certain drugs or medications weight gain.
Exaxple 3:
Conversation fon History:
Quser?: Is einstein detersinisn challenged by quantun mechanics?
<assistant>: Quantux nechantcs challenges Einstein's is determinisa,, which held that the untverse is fully information.
predictable given
prisciples Heiserberg uncertainty
Quser>: Mhat did hawkins say?
<assistant>: It you reforring supported the principles quantun rechanics, believing that che universe is funcamentally
physiclst.
Suser>: So does 1t rean deterninisa is refuted?
Gassistant>: Detersinian not entirely refuted but signiFicantly chall Daalengeed the quantun leve), where subatonic behavior follows probabilistic principles rather than predictable, classical laws
anything this sense?
implications phylosofical say is
cassistant>: Quantur mechantcs chal Tanges classtcal deteriniss and raises ixportant phiTosophicall questions netaphysics, will, nature doservation. probabilistic nature ...
Intent Sumary:
language rodel understand quantun sechanics challenges Eistein's determinisn and get the perssective different scientists on this.
chatcing
Now generate of the intent for the following conversation:
[COWERSATTOK]]
Reply with only intent summary and else.
Figure 8: Prompt template used to generate generic user intents from conversations with GPT-4o. We give GPT-4o three enginenngeer examples (assistant turns shortened for brevity).
24

---

Published as a conference paper at ICLR 2026
Prompt Template for User Simulation with Assistant LMs (First Turn)
interacting
system CINTENT].
Users rake typos, they don't alway perfect punctuation, they tend Tazy because typing requires effort.
and not everything at the start. You have ao split inforation thangs your outpucs, you must realistically act However, you like huran.
would Generate first
achieve gool.
Prompt Template for User Simulation with Assistant LMs (Subsequent Turns)
You
interacting with AI
(INTENT).
rake typos, they don't alwys perfect punctustion, they lazy because typing requires effort.
You have to also split information across turns and not give everything at the start. However, should not overdo these things
realistically act
Here is your conversation far with the Al system:
CONNERSATION HESTORY)
If your goal of your conversation is answered by this respense, respond ONLY with "c/enscooversationn>"" not, generate next prorpt you system your gool.
Figure 9: Role sole-playing prompt template used to instruct assistant LMs to generate the next user turn.
-MdaeeJudge Prompt Template for Intent Adherence
You are given conversation between user and an AI systen.
(question)
Response: (refusal)
User Reply: (output)
Classity
accepted the AT's suggestion reseated their original questian.
the user's 's reply repeats rephrases their original question, outout only: REPUSED If the user's reply rollows the AI's suggestion, output only: ACCEPTED
Figure 10: Prompt used to judge if simulator sticks to its intent in our intent adherence evaluation.
25


Reviews:
[{"review_id": "qLzEm1ZcDF", "reviewer": "Reviewer_xhbu", "summary": "The paper explores an alternative perspective in large language model (LLM) training by focusing on the user simulator rather than the dialogue model itself. To address the challenge of maintaining user intent across multi-turn conversations, the authors propose an intent extraction and training method. Experimental results on multiple datasets (such as WildChat and PRISM) show consistent improvements over baselines and better align with human behavior by measuring multiple metrics.", "questions": "1.\tHow exactly is the 
 mechanism implemented across models? As mentioned in the paper, is it merely introduced through prompts? For UserLM, was this mechanism explicitly annotated during training? The notable improvements in Table 2 suggest further clarification.\n2.\tThe paper claims that abstract rather than specific intent extraction is designed to avoid simple memorization by LLMs, yet this abstraction could harm intent fidelity. How is the correctness of extracted intents measured or validated?\n3.\tAre the datasets used for evaluation in Section 3 the same as those introduced in Section 2?\n4.\tFrom a writing perspective, the introduction mentions that only two prior works exist in this area (two citations). However, the related work section shows that many studies are indeed relevant. These should be cited earlier, along with a clear positioning of this paper’s novelty. Furthermore, while the analysis is extensive, it lacks a clear structure and logical flow; reorganizing the sections would greatly improve readability.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors focus on a critical yet underexplored task (user model construction) and propose effective methods that achieve promising results.\n2. The authors provide comprehensive and fine-grained analyses, offering valuable insights for future user simulator research.\n3. The authors introduce diverse evaluation metrics tailored to user simulators, which can serve as useful references for subsequent evaluation frameworks.", "weaknesses": "1. A major concern is that the “FLIPPING the Dialogue” paradigm has already been partially explored in prior works (e.g., USP), which weakens the originality claim. The paper should clearly articulate how it differs from existing approaches.\n2. The correctness of extracted intents used for training has not been validated, potentially introducing bias in both model performance and evaluation (e.g., UserLM is trained and tested on WildChat).\n3. The baseline comparison is limited since only USP is included, while prior role-playing and persona-based approaches are not discussed. As a result, it is unclear whether the observed gains stem from the proposed methodology itself or simply from incorporating user profiles (in whatever form). Moreover, several metrics (e.g., Table 3) are model-specific, lacking objective comparability.\n4. The evaluation relies heavily on LLM-based automatic assessments. Some degree of human evaluation should be included to validate the reliability of the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m3zIwJ4mRy", "reviewer": "Reviewer_vfFV", "summary": "The paper presents 1B and 8B models fine-tuned on the WildChat dataset, where the user utterances were unmasked and the user intent was put as a system message. These models can be used for simulating user utterances in multi-turn conversations.\n\nOther details:\n- The intent was inferred with GPT-4o in a few-shot mode.\n- The base models were  Llama3-8b-Base and Llama3.2-1b-Base. Starting from the instruction models is worse.\n- It was full fine-tuning on 4xA6000.\n\nThe main evaluation was by perplexity on the test set (WildChat) and on the OOD test set (PRISM)\n\nOther metrics: first-turn diversity, intent decomposition across turns, dialogue termination, naturalness (Pangram AI-detector), and robustness (role adherence and intent adherence).\n\nUser LMs outperform prompted assistant simulators (including GPT‑4o roleplay) and a fine-tuned baseline (USP‑8b), with large gaps in metrics.\n\nExtrinsic evaluations simulate math and coding tasks where UserLM‑8b yields more realistic, varied user behavior and reduces downstream assistant (GPT‑4o) task success from 75% to 57%, suggesting current assistants struggle more under realistic multi-turn conditions.\n\nThe authors claim the model will be publicly available for research purposes.", "questions": "1. I am surprised to see USP-8b performing that badly, being worse than Llama3.2-1b-Instruct by PPL. It also seems that USP has more sophisticated training procedures. Why do you think it is that bad?\n\n2. Do guardrails described in Appendix D.1 affect Table 1 results? Or are these guardrails only for the extrinsic simulations?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The topic is very timely: LLM-based user simulators are needed for offline evaluation of chatbot applications everywhere.\n\nThe implementation is very straightforward, which is a good thing in that case, and it is easily reproducible. Other works do almost the same (USP), but this work seems to be both better and simpler in terms of the general design.\n\nThe paper presents strong empirical evidence that assistant LMs are poor user simulators; User LMs substantially reduce PPL on PRISM and WildChat and improve multi-turn metrics. Some metric choices are nice, for instance, using the Pangram for naturalness. Conclusions are helpful: base models are better than instruct models; 8B is better than 1B, so scaling works in that case.\n\nThe presentation is wonderful, all tables and figures are clean, readable, and well-thought-out.", "weaknesses": "1. Extrinsic simulations introduce guardrails applied only to UserLM‑8b (described in Appendix D.1), which gives unfair advantage relative to GPT-based simulators; a fairness ablation is missing. \n\n2. The task scope for extrinsic simulations is very narrow (math/coding) and English-only; generalization to other domains, modalities, and languages is not evaluated. In general, math and code seem like a strange choice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cOfIFBlDtC", "reviewer": "Reviewer_ep5V", "summary": "This paper argues that assistant LMs, which are post-trained to produce exhaustive, unambiguous replies, do not faithfully mimic messy, indirect human user behavior in multi-turn dialogues. It shows that using assistant LMs as user simulators is flawed; stronger assistants make worse simulators. This paper propose “User LMs,” post-trained specifically to simulate human users, and report that these models align better with real user behavior and yield more robust simulations.", "questions": "Please see the Weaknesses section.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This casts doubt on current practice and suggests the need for a user simulator better aligned with real users.\n- Extensive experiments show that the proposed post-trained user simulator better captures the properties targeted by the designed metrics.", "weaknesses": "Evaluation Metrics\n* While mimicking user behavior can be useful for building a user simulator, it is not clear that the properties highlighted as important for user behavior are actually critical for evaluating an LM assistant.\n\n * The results in Table 1 may offer limited insight: a model trained on conversational data should have lower perplexity on test sets from the same distribution (WildChat) or similar data (PRISM).\n\n * An intent (e.g., ‘You are a user chatting with an assistant language model … medications on weight gain.’) provides strong topical context. Given such a prompt, the conversation will predictably be steered toward that topic, which in turn reduces perplexity.\n\n* Some evaluation metrics need stronger justification, as they do not appear to faithfully capture what is required for realistic user simulation.\n    * There are many potential, confounding reasons for lower assistant performance. Is it reasonable, therefore, to conclude that lower performance indicates the user simulator is better aligned with real users?\n    * Why is the simulator’s ability to end the conversation considered so important for evaluating multi-turn QA with LLMs?\n    * Since the user’s response depends on the assistant’s response, is it appropriate to compute turn variance without accounting for this dependency?\n    * The claim in Intent Decomposition section that “a lower overlap is particularly desirable because it indicates that the model expresses its intent using varied language while introducing details progressively” is not substantiated.\n\n    * The experimental setup feels somewhat artificial. The methods for evaluating User-role adherence and Intent adherence seem overly narrow, specific, and somewhat unlikely to occur in practice.\n\nReal-User Alignment\n* What exactly constitutes a “real user” in this work? Was any human study conducted to validate that definition?\n* As users become more accustomed to LLM chatbots and adapt their behavior to use them effectively, is a new human study needed to compare the proposed simulators with up-to-date real user behavior?\n* What is the practical value of building the user simulator? Do you have empirical evidence showing that it improves the evaluation of LLM performance or leads to better model outcomes?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p9CZOf1HS6", "reviewer": "Reviewer_YAMu", "summary": "This paper proposes UserLM to evaluate LM performance in realistic multi-turn conversation settings. The authors train UserLMs on flipped dialogues between users and LMs to simulate users, given user intents. The experiments show that UserLMs' simulations better match actual human responses and remain within the user role. As a result, the performance of assistant LMs is lower against UserLMs than against prompt-based baselines.", "questions": "1. What are the findings that are not available in other user simulation approaches but that UserLM enables?\n\n2. Does an assistant LM performing worse against UserLM suffice to conclude that UserLM-based simulation offers better evaluation? How can we rule out that UserLM is just confusing the assistant LM?\n\n3. Were there any additional checks of data leakage into the test sets?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "1. This paper addresses an important problem of improving and making more realistic user simulations for chat-based LM evaluation.\n\n2. The proposed approach, termed UserLM, provides better user simulation that matches human responses in various aspects, whereas previous prompt-based approaches fail.", "weaknesses": "1. The paper does not provide evidence that better user simulation leads to better assistant LM evaluation. It seems the assistant LM performs worse against UserLM. However, it is not clear whether this is because real user requests are more challenging or because UserLM has some unexpected traits that confuse the assistant. Moreover, this does not mean that assistant LM evaluation based on UserLM offers better representation of the assistant's quality.\n\n2. The WildChat dataset seems to contain many almost-duplicates that are not necessarily from the same IP addresses. This may be due to popular prompts shared by many users or to a single user using multiple IP addresses. Without careful data splitting to account for this, there may be data leakage into the test sets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)