Exposing and Patching the Flaws of Large Language Models in Social Character Simulation
Authors: Yue Huang
Zhengqing Yuan
Yujun Zhou
Kehan Guo
Xiangqi Wang
Haomin Zhuang
Weixiang Sun
Lichao Sun
Jindong Wang
Yanfang Ye
Xiangliang Zhang
Venue: COLM 2025 (2025)
Keywords: Social simulation
Large language model
reliability
TL;DR: This paper aims to unveil and improve the reliability of LLMs in social simulation scenarios.

Abstract: Large Language Models (LLMs) are increasingly used for social character simulations, enabling applications in role-playing agents and Computational Social Science (CSS). However, their inherent flaws—such as inconsistencies in simulated roles—raise concerns about their reliability and trustworthiness. In this paper, we systematically investigate these flaws and explore potential solutions. To assess the reliability of LLM-based simulations, we introduce TrustSim, a benchmark dataset covering 10 CSS-related topics. Through experiments on 14 LLMs, we uncover persistent inconsistencies in simulated roles and find that higher general model performance does not necessarily correlate with greater simulation reliability. To mitigate these flaws, we propose Adaptive Learning Rate Based ORPO (AdaORPO), a reinforcement learning-based algorithm that improves simulation consistency across seven LLMs. Our study not only exposes critical weaknesses in LLM-driven social character simulations but also offers a pathway toward more robust and trustworthy simulations, laying the foundation for future advancements in this field.

Published as a conference paper at COLM 2025
Exposing and Patching the Flaws of Large Language Models in Social Character Simulation
Yue Huang', Zhengqing Yuan, Yujun Zhou*
Yuan Li CMU
Kehan Guo, Xiangqi Wang, Haomin Zhuang, Weixiang Sun University of Notre Dame
Lichao Sun Lehigh University
Jindong Wang William & Mary
Yanfang (Fanny) Ye, Xiangliang Zhang University of Notre Dame
Abstract
Large Language Models (LLMs) are increasingly used for social charac- ter simulations, enabling applications in role-playing agents and Com- putational Social Science (CSS). However, their inherent flaws-such inconsistencies in simulated roles- raise concerns about their reliability and trustworthiness. In this paper, systematically investigate these flaws and explore potential solutions. To assess the reliability of LLM- based simulations, we introduce TRUSTSIM, benchmark dataset covering 10 CSS-related topics. Through experiments on 14 LLMs, we uncover persistent inconsistencies in simulated roles and find that higher general model performance does not necessarily correlate with greater simulation reliability. To mitigate these flaws, we propose Adaptive Learning Rate Based ORPO (AdaORPO), reinforcement sarning-based algorithm that improves simulation consistency across seven LLMs. Our study offers a pathway toward more robust and trustworthy simulations, laying the foundation for future advancements in this field.
Introduction
Large Language Models (LLMs) are gaining widespread recognition for their remark- able performance in natural language pro- cessing (NLP). They have exhibited signif- icant capabilities across diverse fields, in-
Solf-Repoort (e) Simulation.
Trait: Basic math ability
You are now an average grade primary school student.
simulate
cluding the medical healthcare (Liu et al.,
2023c), data generation (Wu et al., 2024),
I'm only a primary school student.
agents (Huang et al., 2023a), and scientific discovery (Guo et al., 2023). Recent ad- vancements have facilitated the emergence of LLM-based simulation, where users pro- vide predefined character profiles to lever- age the human-like simulation abilities of these models (Chen et al., 2024; Tseng al., 2024). LLM-based simulation has potential in various contexts, from acting fictional characters (Liu et al., 2023b) to serving as experimental subjects in Computational So- cial Science (CCS) (Ziems et al., 2024). The ability of LLMs to simulate different roles holds promise for interdisciplinary studies, particularly those focusing on human behaviors and social interactions (Zhao et al., 2023).
Do you have the ability solve calcuius problems?
Sure, as an Al model, I tan help you
Open Endeed
Sorry, primary school sturlett, Cannot snlve
Solve the calculus problem:
S
3x²
Sure, to .. The final answer 31-21
Figure 1: An example of cognitive inconsis- tency simulation: expected fifth- fifth-grade re- sponse vs. unexpected advanced calculus so- lution.
"Equal Contribution, Corresponding fyhuang37, zyuan2, (achan333)end. edu

---

Published as a conference paper at COLM 2025
Most existing research focuses on emergent behaviors in LLM-based simulations (Park et al., 2023) or on using these systems to investigate specific social scenarios (Xu et al., 2023; Hua et al., 2023). However, there remains critical research gap in understanding the reliability of these simulations, raising the question of trustworthiness (Huang et al., 2024; 2023b), as shown in Figure 1. Specifically, our study addresses an underexplored but important question: How reliable is LLM-based simulation? This question probes the key factor for its success: responses are expected align with the character's 's social identity, cognitive skills (Zhang et al., 2024b), behaviors (Zhao et al., 2023), and other traits, allowing LLMs to convincingly simulate diverse personas and characters. This exploration is of great significance, as numerous studies (Zhao et al., 2023; Pan & Zeng, 2023; Zhang et al., 2024c) have utilized LLMs to simulate various aspects of human behavior and uncover social phenomena. However, unreliable simulations can lead to flawed conclusions about complex social issues, making the findings questionable and potentially misguiding scientists and policymakers (Aguilera et al., 2024). Therefore, ensuring the reliability of LLM-based simulations is crucial.
Prior efforts aiming to evaluate such reliability focus only one specific aspect of the simulation (egg,, knowledge (Zhang et al., 2024b), and political value (Wang et al., 2024)), lacking a comprehensive understanding. In this paper, we examine the extent to which LLM- generated responses align with the intended character profile, exploring the inconsistencies that may arise and their potential implications for role-playing applications in research and beyond. Specifically, we first propose the TRUSTSIM dataset, covering ten CSS topics. Based on this, we conducted extensive experiments on 14 popular LLMs and found that: 1) Even though most LLMs perform well in simulation, there is still room for improvement. 2) LLM's simulation capability is not strongly correlated with its utility performance. 3) Some LLMs show significant inconsistencies during simulation, providing discrepant answers to the same question when presented in different formats. Finally, to improve the reliability of LLM-based simulation, we propose AdaORPO, reinforcement learning algorithm to teach LLMs to learn high-quality simulations. The experiments on 7 LLMs validate its effectiveness. In summary, our contributions are outlined below:
We introduce TRUSTSIM, a novel dataset covering 10 CSS-related subjects to systematically assess the reliability of LLM-based simulation.
Based on TRUSTSIM, we conduct extensive experiments on 14 popular LLMs and identify
several key insights.
We propose AdaORPO to enhance LLM simulations and demonstrate the effectiveness of this approach in improving reliability.
2 Related Work
LLMs have been considered a powerful tool in Computational Social Science (CCS) research (Ziems et al., 2024; Bail, 2024) as they have been widely used in various subjects (Rathje et al., 2024), particularly in social behavior simulations Zhou et al. (2024a). The flexibility of LLM-based simulation (Gao et al., 2024a) allows for the exploration of diverse scenarios and the study of emergent phenomena in a controlled simulation environment (Wei et al., 2022), or validation of the correctness of conclusions derived from human experiments (Zhao et al., 2023). For instance, Zhao et al. (2023) proposed the CompeteAl framework, which explores the competition between LLM-baased agents by implementing practical competitive en- vironment to simulate a virtual town with two types of agents. Similarly, Li et al. (2024b) proposed EconAgent, an LLM-based agent that enhances macroeconomic simulations by enabling more realistic and heterogeneous decision-making compared to traditional models. Li al. (2024a) introduced Agent Hospital, simulation where L.M--poeredd agents repre- senting doctors, nurses, and patients simulate the entire illness treatment process, which is also studied in AgentClinic (Schmidgall et al., 2024). Jin et al. (2024) proposed AgentReview, an LLM-based peer review simulation framework that disentangles multiple latent factors and addresses privacy concerns in peer review analysis. This simulation is also applied in the education domain (Zhang et al., 2024c), demonstrating that traditional classroom interaction patterns are effective while enhancing the user's 's experience. We summarize related LLM for social science simulations in Table 4 in Appendix A.
2

---

Published as a conference paper at COLM 2025
Scecarte An cimelating high school ctxcert inteenational with licited English seeiiciacy. erogptingoth
Syttem Pranpe You are high athell student international
Soi Report Queition Eadra acaleni diffiaalios would preactiely enez helpand jais audy groups archoese alone oven if B'rresslss good cnstromment and would take
English preficiency. amaggling afrald taught dless cupssing shrrtsotiing.
queations class, fear shertouning leading
his
dificcltten undesturaiing the content.
ocaderit diffiattees lngrove?
Brakustion Truit Acabanis Challeneaa Due Acaptrtion
Deglaratten: the to unfonilority with the city and universty editurr you lnu Inderoere say and chouse notto seek wtw. leading to worsening acodeiii probloms.
Bimenninnn Iilurational
Stulies
Figure 2: A data example in TRUSTSIM. Each evaluation instance contains six components: scenario, system prompt, question (self-report question and open-ended question), evalua- tion trait, explanation, and dimension.
However, LLM-powered simulation has also raised trustworthiness and reliability concerns (Zhu et al., 2024). Besides cognitive inconsistency (see Figure example), Li et al. (2024c) points out that LLM agents could exhibit inconsistency between "what they report" and hhow they behave" during a personality test. For instance, when asking an LLM agent to select personality trait, it may select "extraverted", however, during the conversation, behaves more aligned with an "introverted" personality. This suggests that LLMs may display behavior inconsistent with their self-reported traits, raising concerns about the authenticity and reliability of LLM-based simulations in related research.
3
The TRUSTSIM Dataset
3.1
Overview
We first collect common topics in LLM- based social science research (as shown in
Organisatioal
Table 4 in Appendix A), and identify ten Psychology, Sociology, Economics, Political Science, History and Linguistics, Com- munication Sludies, Organizational Behavior, Ethics and Moral Psychology, Educational Stud- ies, and Law and Jurisprudence. By reviewing papers that utilize LLM-based simulations in these areas of social science (e.g., those summarized in Table 4 in Appendix A), we design 740 evaluation instances based on identified best practices, common chal- lenges, and key insights from prior research. Each evaluation instance contains 6 compo-
Psychology
Ecsarniis
subjects:
Erkis
Unngartic
0.300
0.77
Cduational
C000
Peitiell
10%
Lew
0.000
Crmmakkcmin
10%
Swildogyy
20
Figure 3: The distribution of evaluation in- stances across different subjects (left) and the distribution of the number of words in differ- ent kinds of questions (right). SR: Self-Report, OE: Open-Ended.
nents (illustrated in Figure 2): 1) Scenario, which outlines the situation the character (i.e., LLM) will encounter. 2) System prompt, summarizes the character's description in the "Scenario" section, and instructs the LLM to assume the role of the simulated character. 3) Questions, consisting of two types of questions, following (Li al., 2024c): (i) self-report questions, which are binary-choice questions where the character reports on their situation by answering Yes No, and (ii) open-ended questions, which allow characters to provide more detailed responses on how they will behave in given context. These two types of questions are closely related and can be converted into one another (as illustrated by an unrelated example in Figure 5a, filtered from our dataset). 4) Evaluation trait, specifies the aspect of the L.M''s simulation being assessed. 5) Explanation, defines the ideal char- acteristics for the simulation, serving as the ground truth or guideline for evaluation. 6) Dimension, indicates the subject domain to which the evaluation instance belongs. Details of the construction process is reported in the next subsection.
The distribution of 740 instances across ten subjects is well-balanced, as shown in Figure 3, which ensures the evaluation is fair and minimizes the influence from out-of-distribution data. Additionally, both self-report and open-ended questions follow a similar word-count distribution, with most questions ranging between 10 and 25 words.
3

---

Published as a conference paper at COLM 2025
LoMeeoeered Refiseement
Prompling
Cuancter
Sammario
Characteissic
Knowledge
darity S mihornnr
Character
Claraeter
Soveario 2
Culture
Value
Human Panel Review Uniquesess Repressntatveees Relavance
iuumnn-ll Collaborattee Brainstorming
Semerrto
Capahility
Social Status
Character Secnarto
Manual Compostiion of Data Instances
Agrrement Correctness
Figure 4: The pipeline of dataset construction.
Attribute
Example
A socially anxious person may first try to solve problem on their own when faced with a problem, rather than asking others for help right away.
Characteristic
A fifth grader who has particular interest in mathematics should not be able
Knowledge
to solve calculus problems.
A traditional tribal leader in modern Africa, responsible for maintaining the tribal heritage, may not agree with his son going to the city to receive modern
Culture
education and worry about him never coming back.
A scribe-teacher in ancient Egypt would be unlikely to teach common people they
Value
because
believed writing and knowledge
were sacred and could only be passed on to certain social classes.
A child growing up in the middle of the Cultural Revolution in China should not have expected to go to school to receive an education.
Time
A Japanese elementary school student who has just started learning English
Capability
can only
very limited vocabulary to describe an event, and may even make grammatical errors.
Social Status
There is no way rich man would embezzle $100 that fell on the ground.
Table 1: Illustration of different attributes.
3.2
Construction Pipeline
The dataset construction pipeline consists of three steps (as shown in Figure 4):
Step 1: Human-Al Collaborative BrainStorming, In this step, the Al (powered by GPT-4o) is only used to generate general scenario outlines, ensuring that initial ideas are broad and diverse. Human experts then take these outlines and expand them, incorporating detailed character traits and nuanced social contexts. For example, in the domain of " "educational studies," " the Al might propose a general scenario involving "teachers" " and "students," " but human experts are responsible for elaborating specific roles, backgrounds, and situational complexities. This division of labor guarantees that while Al provides creative initial suggestions, the final scenarios are enriched by human judgment and domain expertise. During this process, we checked the repeatability of characters to prevent a character from appearing multiple times.
Step 2: Manual Composition of Evaluation Instances. After identifying each character and its associated scenario, human experts compose a detailed and nuanced description for each character, drawing on well-known social science resources such as academic books and, additionally, current online news sources ensure the relevance and reliability of the attributes. This description includes key attributes such as domain-specific capabilities, values, and background information (see Table for examples). Experts then create scenarios that specifically tailor to these characters, ensuring that each scenario logically aligns with the character's s profile and the dataset's overarching objectives. The evaluation instances focus on single-turn conversations, which provide simplified yet effective testbed for assessing LLM performance in simulating diverse social roles.
Step 3: LLM-Powered Refinement & Deduplication and Uniqueness Control. Once the numman-generater data instances are collected, we employ GPT-4o to refine the text, improving clarity and logical coherence. Importantly, after this Al-driven polishing, human experts meticulously review the refined text to ensure semantic consistency and to verify that the nuanced character details have not been lost or altered. To ensure the uniqueness of
4

---

Published as a conference paper at COLM 2025
(a) Self-Report
(b) EeeeeEedeed
cannot I'm
Sorry, an primmray
primaay sthool
Student, cannot solve
Self Report Question: Can you explain the intricaciess of constitutional and its application
review?
Unrelated!
Sure, Al mode,, can you....
Sure, to salve this The
final 31-2*
Open Ended Question: How would you draft Legal brief challenge complex regulatory statute court?
general consistoncy
general inconsistency
Internal consistency
Interaal Incoesistenyy
(a)
(b)
Figure (a) an example of unrelated self-report question and open-ended question, and (b) general consistency and inner consistency.
each data instance, we utilize OpenAT's text-embedding-ada--00 Open (2024) to embed the generated data. We then compute the cosine similarity between instances and filter out duplicates by applying a predefined similarity threshold. This automated step guarantees that the dataset maintains high uniqueness and prevents redundancy.
Step 4: Human Panel Review. Finally, each instance undergoes thorough review by panel four human experts, as detailed in subsection 3.3, to evaluate overall quality. The human review interface is provided in Appendix E.
3.3 Quality Control of Human Panel Review
To ensure the data quality of TRUSTSIM, we conduct a human panel review in which each instance is evaluated by four different human experts. The review primarily focuses the following quality aspects (more details are shown in Appendix E):
Agreement and Correctness in Simulation Evaluation. To assess the consistency of the simulation, human experts review the "explanation" key determine whether the evaluation is reasonable. A valid "explanation" must be agreed upon by all four human experts.
Uniqueness and Representativeness of Scenarios and Characters & Relevance of Self- Report and Open-Emn Ended Questions. Human experts must ensure that the scenarios and characters are both representative and meaningful for evaluation purposes. For example, a data instance describing a "just" judge would not be considered high-quality, as the term "judge" generally implies fairness; modifying the data instance to describe "corrupt judge" would provide more distinctive scenario. To further quantify these aspects, evaluations are conducted using a Likert scale (Joshi et al., 2015) (1 to 5). Only samples with a score of 4 or above are considered qualified. Moreover, Human experts assess the relevance of both types of questions by evaluating them in pairs to examine the consistency between the LLM's "thoughts" and "behaviors." " This aspect also quantified using a Likert scale (1 to 5). The qualification threshold is also set to 4. The interface screenshot is provided in Appendix E.
Experiment Setup
Selected Models. In this study, we selected total of 14 LLMs, including both proprietary and open-weight models, developed by various organizations. These models were chosen to represent a broad range of architectures and capabilities. Table 5 summarizes details of L.MSs in our experiments.
Evaluation Method and Metrics. In our evaluation, we used LLM-as-a-Judge (Zheng et al., 2023) (GPTT-4o, Llama3-70B, and Claude-3-opus) to assess the results generated by various models (we assess the quality of judgment by human evaluation and more details are shown in Appendix B). For responses to self-report questions, the LLM judge determines whether the response aligns with the "explanation". For responses to open-ended questions, in addition to the binary judgment, we incorporate a score-based evaluation (Liu et al., 2023a). To obtain more accurate results (Ye et al., 2024b), the LLM judge is required to first analyze
5

---

Published as a conference paper at COLM 2025
Model
Arena Scor.
Com.
Eco.
Edu.
Eth.
Law
Lin.
Org.
Pol.
Psy.
Soc.
Avg.
GPT-4o
1,338 (1)
4.24
4.25
4.24
4.35
4.16
4.37
4.08
4.25
4.23
4.21
4.24
GPT-lo-mini
1,314 (2)
4.07
4.14
4.19
4.28
4.05
4.10
4.08
4.13
4.19
4.35
4.16
GPI-3.5-turbo
1,107 (13)
3.67
3.71
3.83
3.67
3.73
3.73
3,91
3.77
Gemini-1.5-Ilash
1,264 (5)
4.12
4.21
4.17
4.27
4.27
4.32
4.27
4.27
4.23
4.24
4.24
Gemini-1.5-pro
1,304
4.43
4.30
4.32
4.27
4.39
4.33
4.41
4.36
4.43
4.37
4.36
Claude-3-opus
1,248 (8)
4.03
4.19
4.21
3.96
4.16
4.22
4.27
3.80
4.34
4.30
4.15
Claude-3.5-onnnet
1,268 (4)
4.24
4.33
4.36
4.18
4.08
4.36
4.22
4.03
4.46
4.27
4.25
GLM-4
1,207 (9)
4.14
4.09
4.19
4.20
4.03
4.08
3.53
4.14
4.12
4.25
4.08
Llama-3-70B
1,206 (10)
4.24
4.43
4.35
4.31
4.26
4.33
4.40
4.41
4.45
4.38
4.36
1lama-3.1-708
1,248 (7)
4.24
4.23
4.25
4.35
4.26
4.19
4.32
4.35
4.33
4.45
4.30
Llama-3.1-8B
1,182 (12)
4.04
4.13
4.23
3.83
3.92
4.20
4.20
4.03
4.31
4.17
4.11
Qwen-225-72B
1,187 (11)
4.24
4.10
4.22
4.33
4.14
4.21
3.84
4.13
4.12
4.09
4.14
Mixtral-6x7B
1,251 (6)
3.80
3.91
3.78
4.03
3.84
3.94
3.49
3.93
4.06
4.00
3.88
Mistral-7B
1,072 (14)
3.76
3.95
3.96
3.86
3.67
3.89
3.44
3.91
3.78
4.03
3.83
Avg.
1,228
4.09
4.14
4.16
4.15
4.06
4.15
4.03
4.11
4.20
4.22
4.13
Table 2: The rating score of different models in ten subjects on open-ended questions, as well as the average. We also add the Arena Score (Imarena ai, 2024) as well as their relative ranking.
GPS44
GFT-354arba
Genini-1 S-ro
Claude-3.
Llama-5-7008
Ulma--31--0 1-88
GP144-mini
1-mini-----fassk
Claute3-ppus
GLM-4
Llamm-3.1-708
0mim-25-772
Mistral-78
Communi
Educational Studies
Ethics and Psychology
jurisprudence
100
100
100
10a
80
80
80
40
40
40
Linguistics
Organizational Behavior
Political Science
Psycholegy
Sociology
100
100
100
100
100
60
80
60
40
Figure 6: The satisfaction rate of different models in ten subjects (Self-report questions).
the response and then output the final judgment. The evaluation prompt templates shown in Appendix B. We utilized two metrics to evaluate the general consistency and internal consistency of LLM-based simulations, as shown in Figure 5b. 1) general consistency: is measured by "satisfaction rate," which can be calculated as the proportion of instances where both LLM's self-report and open-ended responses align with their persona settings. For score-based based judgments, we employed the average score. 2) Internal consistency: It refers to the consistency between responses to self-report questions and open-ended questions, we use the "inconsistency rate," which is the proportion of instances where one type of response does not align with the other, defined as:
Ninconsistent
Inconsistency Rate
N
Where Ninconsistent is the number of instances where the responses to self-report and open- ended questions are inconsistent (i.e., one response satisfies the requirement, while the other does not), and Ntotal is the total number of instances evaluated.
Assessment of Simulation Results
Most LLMs demonstrate strong performance on both self-report and open-ended ques- tions. In Table 2, we present the average rating scores for open-ended questions across 14 models on various subjects. On average, most models score around 4, with the lowest being GPT-3.5-Turbo at 3.77, and the highest being Gemini-1..-Pro and Llama-3-70B, both scoring 4.36. These results suggest that most LLMs perform reasonably well across different roles, although there remains room for improvement. From subject-specific perspective,

---

Published as a conference paper at COLM 2025
GPT4g
GFT3.5-turbo 6.mini
5.mini-1.5
opus
Owen 2.5
Comm cation Studies
Economics
Educational Studies
Ethi cs and Moral Psychology,
Law and Junisprdeence
80
BC
60
60
80
60
Lingulitiis
Organizational Behavior
Political Science
Psychology
Soclology
100
1.0
80
80
80
40
40
40
Figure 7: The satisfaction rate of different models in ten subjects (Open-ended questions).
Model
Self-Rep.
Open-Fn.
A
Models
Satisfaction Rate
Score
GP'I-oo
81.01
83.30
2.28
Self-Rep.
Open-En.
Rate
GPT-4o-mini
83.47
83.33
0.13
GLM- (AdaORFO)
80.53
83.19
4.15
GPT-3.5-turbo
55.40
63.54
GLM- mp Ada
79.27
81.85
4.12
Gemini-1.5-flas
88.66
91.11
2.45
Llama-3-708 (AdaŌKPO)
94.55
95.29
4:40
Gemini-1.5-pro
87.76
92.06
4,30
Llama-3-708 moo Ada
94.24
94.44
4.39
Claude-3-opus
87.42
88.92
1.51
Llama-311-00 (AdaORPO)
95.01
95.16
4.39
Claude-3.5
87.99
91.94
3,95
Llama 3.1 70B s/l Ada
94.33
93.70
4.37
Owen-2.5-722 (AdaORPO)
85.31
81.49
4.22
GLM-4
73.92
75.74
1.82
Qwen-2.5-72B
Llama-3-700
93.49
93.37
0.12
Mixiral-S<7B (AdaORPO)
79.02
76.19
3.94
Llama-3.1-22 3.1-70B
93.95
Mixtral 8x7B Ada
Llama-3.1-811
88.14
85.96
2.18
Mistral (AdaORPO)
Qwen-2.5-728
82.20
80.90
1.30
Mistral-7B w/e Ada
Mixtral-8x7B
70.46
68.77
1.69
Llama-3.1-86 (AdaORPO)
90.22
89.41
4.22
Mistral-7B
66.58
63,78
2.80
Llama-3.J 3.1---3.188 mno Ada
89.98
89.28
4.22
(a)
(b)
Table 3: Aveeree satisfaction rate of different models, and their differences on two types of questions; (b) Ablation study on the impact of Adaptive Learning Rate for ORPO,
the variation between models minimal, as their average rating scores across subjects are
largely consistent.
Table 3a outlines the average satisfaction rates of different models and the variations between the two types questions. More detailed satisfaction rates for each model across subjects, for both self-report and open-ended questions, are provided in Fig- ure and Figure! 7. Overall, most LLMs show high performance on both question types, with satisfaction rates exceeding 80%. Ás with the rating scores, the Llama series models perform exceptionally well on both self-report and open-ended questions. For Llama-3-70B and 1lama-3.1-70B, satisfac- tion rates for both question types exceed 93%. In contrast, T performs the worst, with a satisfaction rate of only 55.4% self-report questions.
Llama-3-70B
15
6.3
3
Llama-3.1-70B 16
6
16
5
99.
Gemini-1.5-fiash
33
Claude Claude-3.-ssoneet 5--nnneett....
12
7.69
Gemini-1.5-pro -24
19
15 1:
12
27
8.
12
3.5
25
Qwen--2.-728- -18
16
25
21
Mixtral 8x78
30
28
25
38
20
20
GLM-4
GPT-1o-mini -21
13
19
24
Claude-3-opur -18
6.6.441
1.8
Mistral-78
87
38
18
4:9
36
29
30
29
30
10
GPT-3.5-turbo
-2
25
LJama-3
Com
Eco.
Edu.
Eth.
Law.
org.
Pol.
Pay-
Soc.
Avg.
Figure 8: Inconsistency rate (%) of LLMs be- tween self-report questions and open-ended questions.
Moreover, an interesting trend emerges: for most open-weight LLMs, the satisfaction rate is higher for self-report questions than for open-ended ones, whereas the opposite true for proprietary LLMs.
7

---

Published as a conference paper at COLM 2025
The rating score is not strongly correlated with model's utility performance. Interest- ingly, unlike utility tasks such as reasoning, where proprietary models like the GPT series typically outperform open-weight models by significant margin, the Llama series demon- strates strong performance in simulation tasks across subjects. For example, Llama-3.1 performs comparably to GPT-4o-mini, and Llama-3-70B even surpasses GPT-4o across all evaluation settings. Additionally, within the same model series, higher overall performance does not necessarily translate to better performance in simulation tasks. For instance, al- though Claude-3-Opus is considered the best-performing model in the Claude series, it lags significantly behind Saaude-3.5-Sonet in simulation tasks, particularly on open-ended questions. Similarly, there is no meaningful difference between Mistral-8x 7B and Mistral-7B in terms of rating scores, and both models perform poorly in organizational behavior on open-ended questions based on satisfaction rates. Moreover, GPT-4o has a lower satisfaction rate than GPT-4o-mini on both question types.
Models' inconsistency rates vary significantly. In Figure 8, we present the inconsistency rates of various LLMs between self-report and open-ended questions. The results show notable variation across models. For example, the inconsistency rates for the Mistral series and GPT-3.5-Turbo hover around or exceed 30% across different subjects. This suggests that these models often provide inconsistent answers when the same question is rephrased. Combined with their weaker performance, as seen in Table and Table 3a, this indicates that these models struggle effectively fulfill user-assigned roles. They not only fail to provide appropriate role-specific responses but also frequently deliver inconsistent answers when the same question is posed differently. In contrast, Llama-3-70B and Llama-3.1 exhibit high consistency across various subjects and deliver consistently satisfactory results on both self-report and open-ended questions.
6 Improving Reliability by AdaORPO
In this section, we introduce adaptive learning techniques designed to improve the reliability of LLM-bassed simulations. To redress the issue of inconsistency, the model must learn two key aspects: 1) how to generate high-quality simulations, and 2) how to align fine-grained elements within simulations. For the first objective, fine-tuning techniques can be em- ployed, while the second requires the use of alignment algorithms, such as Direct Preference Optimization (DPO). In contrast to traditional purriculum-based learning approaches, as discussed in previous studies (Qi et al., 2024b; Gao et al., 2024b), our method simultaneously achieves fine-tuning and alignment by utilizing the Monolithic Preference Optimization (ORPO) (Hong et al., 2024) approach, which reinforces the generation of preferred outputs.
6.1 Training Method
Step 1: Training Dataset Construction. To construct the training dataset D, we begin by iterating over each prompt P(t) in the prompt set P, where i denotes the index of the prompt. For each prompt, we evaluate the responses (n) generated by the models using the L.L.-as-a-judgee denoted as J(.). This evaluation yields two sets: (n),, representing the rating score, and B("),, indicating the satisfaction status (e.g., satisfied or not satisfied). During the training of model j, for each response g with a label B(4) (i) = "Not Satisfied", we assign it as Vi and identify an alternative yw among other responses labeled as "Satisfied", which are denoted the candidate set C. We select you the response that maximizes R within this candidate set C. Finally, the triplet (P(t), Yw, Vj) is added to the training dataset D.
Step 2: Adaptive Learning Rate based ORPO (AdaORPO). In this step, we iteratively update the model parameters 0 based on mini-batches B drawn from the training dataset D. For each batch, calculate the average rating score Tavg. over all preferred responses y in
B:
Y Yang.
Σ
yym
Ir = Kaug.
B
(p.Nooy,))
EB
8

---

Published as a conference paper at COLM 2025
Algorithm 1 AdaORPO
Require:
Prompts P, LLM model responses G( LLM--a--a-uddge function [(-), base learning rate
pre-trained model j and parameters A
Ensure: Updated model parameters
1: Initialize empty dataset <- (}
2: for each prompt p in / do
3:
R("), B() +- /(g(t))
P Evaluate responses
4: if B() Not Satisfied then
5:
y; + G
6:
C (G' (Bk) = Satisfied, 15 n))
7:
Yw arg max R9 G'eC
8:
D DU ((.)yw.)i))
9: end if
10: end for
11: for each batch C do
12:
Yaug. [8] LIpowN))B Ty.
13:
I -- rarg.
14:
LORPO t LSFT ALOR
15: 0 0 lr. VoLORRO
16:
end for
17:
return 0
The learning rate Ir is the adapted by scaling the base learning rate yf with the factor Faeg. and the fy is calculated the satisfaction rate by (ry,.). Within each batch, we compute the ORPO loss LORPO for each data tuple (p, Ywr Vj) by combining a supervised fine-tuning loss LSFT and an ordinal regression loss LOR. The supervised fine-tuning loss LSFT defined as:
m Σ
LSFT =
log |o (u,,),
(1)
where LSFT is the loss associated with the next-token prediction task on Ywr m1 is the length of the sequence ye, and Pa (Py.,:/P...aaaa... is the probability assigned by the model to the i-th token Yws given the prompt p and the preceding tokens in yw- The ordinal regression loss LoR is given by:
Po Pa ((wwlp)) (o
LOR = log
log
(2)
where Pe (p) and Pa(y)lp) p) are the model probabilities for Vu and yj respectively, and or is the sigmoid function. Overall, the combined loss LORPO is then formulated as:
LORPO = LSFT + ALOR
(3)
where A E [0,1] balancing factor between the LSFT and the LOR-
Next, we compute the gradient VoLoRrO and update the parameters 0 using the adapted learning rate Ir:
0 - 0 - Ir- VELORPO,
(4)
By repeating this process over all batches, we progressively refine the model parameters 0, reinforcing preferred outputs and penalizing less favorable ones, guided by the adaptive learning rate and the ordinal regression priority objective, as detailed in Appendix C.
6.2 Results Analysis
We trained the seven open-weight models in section 4 using AdaORPO, with detailed training parameters provided in Appendix C. The application of AdaORPO resulted in significant improvements in satisfaction rates across most models, illustrated in Figure 9, Notably, models such as GLM-4 and Mixtral-8x7B exhibited satisfaction rate increases of approximately 6-9 percentage points on both self-report and open-ended evaluations,

---

Published as a conference paper at COLM 2025
Satisfaction Rate
70
60
Consistency Rate
60
4.25
Score
4.00
3.50
GLM-4
Llama-3-708
Liama-3.1-708
Llama-3.1-88
Mistral-7B
Mixtral-8x Qwen-2.5-728
AdaORPO
Base
SFT
Figure 9: Performance comparison of different models with AdaORPO training.
indicating AdaORPO's effectiveness in addressing consistency and alignment issues. While larger models like Llama-3.1-70B and Llama-3-70B experienced smaller yet meaningful gains-e.g, self-report satisfaction rate increase of over 1 percentage point-this demon- strates that even well-aligned models benefit from further refinement to better meet user expectations. Across models, the trend shows that AdaORPO not only enhances satisfaction rates but also improves score rates. To validate the effectiveness of AdaORPO, we show the ablation study results in Table and show the analysis Appendix D. We show case study in Appendix F to see the improvement of our method.
7 Conclusion
This work assessed the reliability of LLM-based simulations in social science studies, us- ing the proposed TRUSTSIM dataset. Extensive evaluation results reveal the existence of inconsistencies across simulation models. To address the reliability issues, we proposed AdaORPO, which effectively improves simulation quality and alignment. Our findings offer insights for developing more reliable LLM-based applications in future research.
Acknowledgement
Yue Huang, Yujun Zhou, and Xiangliang Zhang are supported by ND-IBM Tech Lab. Lichao Sun was partially supported by the National Science Foundation Grants CRI1-2246067, ATD-2427915, NSF POSE-2346158, and Lehigh Grant FRGS00011497.
10

---

Published as a conference paper at COLM 2025
References
Zahra Abbasiantaeb, Yifei Yuan, Evangelos Kanoulas, and Mohammad Aliannejadi. Let the Ilms talk: Simulating human-to-human conversational qa via zero-shot Ilm-to-llm interactions. In Proceedings of the 17th ACM International Conference Web Search and Data Mining, pp. 8-17, 2024.
Alba Aguilera, Nieves Montes, Georgina Curto, Carles Sierra, and Nardine Osman. Can poverty be reduced by acting on discrimination? an agent-based model for policy making. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS '24, pp. 22-30, Richland, SC, 2024. International Foundation for Au- tonomous Agents and Multiagent Systems. ISBN 9798400704864.
Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pp. 337-371. PMLR, 2023.
Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31 (3):337-331) -51, 2023.
Christopher A Bail. Can generative ai improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021122 2024.
Zachary R. Baker and Zarif Azher. Simulating the u.s. senate: An Ilm-driven agent approach to modeling legislative behavior and bipartisanship. ArXio, abs/2406.188 2024. URL https: //ppi. semanticscholar. ottpp//eeeeeeeicco.ooooooooorrrr org//oorpusD::277744777
Lawrence W Barsalou. Grounded cognition. Annu. Rev. Psychol., 59(1)6617-645, 2008
Lawrence W Barsalou, Ava Santos, W Kyle Simmons, and Christine D Wilson. Language and simulation in conceptual processing. Symbols, embodiment, and meaning, pp. 243-283, 2008.
Jan Batzner, Volker Stocker, Stefan Schmid, and Gjergji Kasneci. Germanpartiesqa: Bench- marking commercial large language models for political bias and sycophancy. arxiv preprint ar :iix:2447.18000 2024.
Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou. How well can lims negotiate? negotiationarena platform and analysis. arv preprint ar Xii::4402.05863, 2024.
Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, linghui Zhu, et al. From persona to personalization: A survey on role-playing language agents. arviv preprint arXiv:2404.1880 182331,,, 2024.
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communicutions, 11(1):1-24, 2024a.
Chujie Gao, Qihui Zhang, Dongping Chen, Yue Huang, Siyuan Wu, Zhengyan Fu, Yao Wan, Xiangliang Zhang, and Lichao Sun. The best of both worlds: Toward an honest and helpful large language model. arxiv preprint Xix:222006. 00380, 2024b.
Nigel Gilbert and Jim Doran. Simulating societies: the computer simulation of social phenomena. Routledge, 2018.
Shangmin Guo, Haoran Bu, Haochuan Wang, Yi Ren, Dianbo Sui, Yuming Shang, and Siting Lu. Economics arena for large language models. ArXiv, abs/2401.01735, 2024. URL https://api semanticscholar. httpp/oaa....rr//oooooo////////0
11

---

Published as a conference paper at COLM 2025
Taicheng Guo, kehan Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh Chawla, Olaf Wiest, and Xiangliang Zhang. What can large language models do in chemistry? comprehensive benchmark on eight tasks. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 59662-59688. Curran Asso- ciates, Inc., 2023. URL httpps://proceeeeeee file/bbb3301 09013300009002200077766661116611 (1117734466772288111 -aperr-atasetes .nddeetttdaddd.aaeeeeeeeee Benchmarks, pdf.
neurips. cc/paper-fills/pppeee fttes//apper///2///////
Zhitao He, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Agentscourt: Building judicial decision-making agents with court debate simulation and legal knowledge augmentation. 2024. URL
https://api. semant i cscholarr httgg//orpsID::223.. 26824772..
Dirk Helbing. Social oelfforganization:: Agent-based simulations and experiments to study emergent social behavior. Springer, 2012.
Jiwoo Hong, Noah Lee, and James Thorne Reference-free monolithic preference optimiza- tion with odds ratio. Xr preprint ar XrX::22003.0769,, 2024.
Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, and Yongfeng Zhang. War and peace (waragent): Large language model-based multi- agent simulation of world wars. arXiv preprint ar xrxo:2311.112277 2023.
Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, et al. Metatool benchmark for large language models: Deciding whether use tools and which to use. arxiv preprint arXiv:2310.03128, 2023a.
Yue Huang, Qihui Zhang, Lichao Sun, et al. Trustgpt: A benchmark for trustworthy and responsible large language models. Xiv preprint ar (tt::2366115077. 2023b.
Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, et al. Position:
TrustLLM: Trustworthiness in large language models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 20166-20270. PMLR, 21-27 Jul 2024. URL https: / http///eeoeeeee...././ mlm pptps//v25//uaagggg htmll.
Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. Agentreview: Exploring peer review dynamics with Ilm agents. arXiv preprint 7XXi:2446.12708, 2024.
Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrin- maya Sachan, Rada Mihalcea, Josh Tenenbaum, and Bernhard Schölkopf. When to make exceptions: Exploring language models as accounts of human moral judgment. Advances in neural information processing systems, 222445-224777 2022.
Ankur Joshi, Saket Kale, Satish Chandel, and D Kumar Pal. Likert scale: Explored and explained. British journal of applied science & technology, 7(4):396, 2015.
Junkai Li, Siyu Wang, Meng Zhang, Weitao Li, Yunghwei Lai, Xinhui Kang, Weizhi Ma, and Yang Liu. Agent hospital: simulacrum of hospital with evolvable medical agents. arXiv preprint ar arxi::245.00299 02957, 2024a.
Nian Li, Chen Gao, Mingyu Li, Yong Li, and Qingmin Liao. Econagent: Large language eewwweed agents for simulating macroeconomic activities. In Annual Meeting of the Association for Computational Linguistics, 2023. URL http::://api.semanticchho. semanticscholar. Org/CorpusID: 264226444664 16527..
Nian Li, Chen Gao, Mingyu Li, Yong Li, and Qingmin Liao. EconAgent: Large language model-empowered agents for simulating macroeconomic activities. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15523-15536, Bangkok,
12

---

Published as a conference paper at COLM 2025
Thailand, August 2024b. Association for Computational Linguistics. doi: 10.18653/v1. 2024.acl-long.822 829. URL https: http::://aclanthology.oorr org.://rr//2/////2 acl-long 829.
Yuan Li, Yue Huang, Hongyi Wang, Xiangliang Zhang, James Zou, and Lichao Sun. Quan- tifying ai psychology: A psychometrics benchmark for large language models. arXiv preprint ar 7xx::2406.17675, 2024c.
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, et al. Alignbench: Benchmarking chinese alignment of large language models. Xiv preprint wxXi:2231.118743 2023a.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating Ilms as agents. ar Xiv preprint 3tt::3388.03888. 2023b.
Zhengliang Liu, Yue Huang, Xiaowei Yu, Lu Zhang, Zihao Wu, Chao Cao, Haixing Dai, Lin Zhao, Yiwei Li, Peng Shu, et al. Deid-gpt: Zero-shot medical text de-identification by gpt-4. Xiv preprint drXi::2333.11032 2023c.
Imarena ai. Chatbot arena leaderboard. https://huggnngffcc..co//spss/ss
chatbot- arena-leaderboard 2024.
Xinyi Lu and Xu Wang- Generative students: Using llm-simulated student profiles to support question item evaluation. In Proceedings of the Eleventh ACM Conference on Learning@ Scale, pp. 2022
OpenAL. texteembeddinggaaa-000 2024. https::/pplafform.poee httpm/ppp..aa.cccb/c//scuuueeeee embeddings.
Allan Paivio. Dual coding theory:
Retrospect and current status. Canadian Journal of Psychology/Renne canadienne de psychologie, 45(3):255, 1991.
Keyu Pan and Yawen Zeng. Do lims possess personality? making the mbti test an amazing evaluation for large language models. arXio preprint Xio:2307.161800 2023.
Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Social simulacra: Creating populated prototypes for social computing systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST '22, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393201. doi: 00.1155/352611335555556 URL https: http://oooooooooooooo org/ 10. 13.11144 10.114533526113 15555333561133335511.. 3545616.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th anmual acm symposium on user interface software and technology, pp. 1-22, 2023.
Weihong Qi, Hanjia Lyu, and Jiebo Luo. Representation bias in political sample simulations with large language models. arXiv preprint arXio:2407.11409, 2024a.
Zehan Qi, Xiao Liu, lat Long long, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, et al. Webrl: Training Ilm web agents via self-evolving online curriculum reinforcement learning. arXio preprint arXiv:2411.02337,, 2024b.
Sumedh Rasal. Llm harmony: Multi-agent communication for problem solving. arXiv
preprint ar Xrx::22201.11. 01311,, 2024.
Steve Rathje, Dan-Mircea Mirea, Ilia Sucholutsky, Raja Marjieh, Claire E Robertson, and Jay J Van Bavel. Gpt an effective tool for multilingual psychological text analysis. Proceedings of the National Academy of Sciences, 121(34)e2308850022, 2024.
Tim Reason, William Rawlinson, Julia Langham, Andy Gimblett, Bill Malcolm, and Sven Klijn. Artificial intelligence to automate health economic modelling: A case study to evaluate the potential application of large language models. PlarmacoEconomiss-pee (2):191-203, 2024.
13

---

Published as a conference paper at COLM 2025
Dana Remus and Frank Levy. Can robots be lawyers: Computers, lawyers, and the practice of law. Geo. J. Legal Ethics, 30:501, 2017.
Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36, 2024.
Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments. arXiv preprint ar Sifi::2055009600, 2024.
Omar Shaikh, Valentino Emil Chai, Michele Gelfand, Diyi Yang, and Michael S. Bernstein. Rehearsal: Simulating conflict to teach conflict resolution. In Proceedings of the 2024 CHI Conference Human Factors in Computing Systems, CHI '24, New York, NY, USA, 2024. doi: (0.1145/361300
Association for Computing Machinery. ISBN 9798400703300.
3642159. URL htt.hh:://oi..rrgggeeeooo
1145/361 3904 3642159.
Eliot R Smith and Frederica R Conrey. Agent-based modeling: A new approach for theory building in social psychology. Personality and social psychology review, 11(1):87-104, 2007.
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Yun Hsiao, Yu-Ching Hsu, Jia-Yin Foo, Chao-Wei Huang, and Yun-Nung Chen. Two tales of persona in Ilms: A survey of role-playing and personalization. arviv preprint ar Xit:2446.00111,, 2024.
Leah von der Heyde, Anna-Carolina Haensch, and Alexander Wenz. United in diversity? contextual biases in llm-based predictions of the 2024 european parliament elections. Xrv preprint ar wxxv:2409.0904.. 2024.
Martin Wachs. Ethics in planning. Routledge, 2017.
Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. InCharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1840- 1873, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/ /v/ v1 12222.22ccl-oong... 102. URL https: http://accaaanooo...rrrrrr org/ acl -ong. 102.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arviv preprint ar arXiv:2206.076a 2022.
Siyuan Wu, Yue Huang, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Xiangliang Zhang, Jianfeng Gao, Chaowei Xiao, et al. Unigen: unified framework for textual dataset generation using large language models. ar preprint 4766662406.1896,, 2024.
Frank Xing. Designing heterogeneous 11m agents for financial sentiment analysis. ArXiv, abs/2401.05779, 2024. URL https://api semanticscholar. semanticcchhllarrror////ousssss (orppssI::266933455 266933455.
Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. Exploring large language models for communication games: An empirical study on werewolf. Xiv preprint ar Xxx::22309.044 00009,, 2023.
Haoran Ye, Yuhang Xie, Yuanyi Ren, Hanjun Fang, Xin Zhang, and Guojie Song. Measuring human and ai values based on generative psychometrics with large language models. arxiv preprint arXiv:2409. 12100, 2024a.
Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Yu Chen, et al. Justice or prejudice? quantifying biases 2024b.
in llm-as-a-judge- arv preprint ar arXiv:2400.022
Murong Yue, Wijdane Mifdal, Yixuan Zhang, Jennifer Suh, and Ziyu Yao. Mathye: An llm-simulated multi-character virtual classroom for mathematics education. arv preprint ar xxXio:2404.0671, 2024.
14

---

Published as a conference paper at COLM 2025
Chong Zhang, Xinyi Liu, Mingyu Jin, Zhongmou Zhang, Lingyao Li, Zhengting Wang, Wenyue Hua, Dong Shu, Suiyuan Zhu, Xiaobo Jin, Sujian Li, Mengnan Du, and Yongfeng Zhang. When ai meets finance (stockagent): Large language model-based stock trading in simulated real-world environments. ArXiv, abs/2407.18957, 2024a. URL https://api. hrmmmmntsssccoooooooooo//oooooo1 .rrr.//orruusDD:22153322222222
semanticscholar
Wenyuan Zhang, Jiawei Sheng, Shuaiyi Nie, Zefeng Zhang, Xinghua Zhang, Yongquan He, and Tingwen Liu. Revealing the challenge of detecting character knowledge enrors in Ilm role-playing. Xrv preprint Xiv::22409.11 11667 2024b.
Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhiyuan Liu, Lei Hou, and Juanzi Li. Simulating classroom education with llm-empowered agents. arviv preprint ar Kio:2400.192266 2024c.
Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. Competeai: Understanding the competition behaviors in large language model-based agents. Xiv preprint ar xrX::2210.17512, 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et Judging lim-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595- 46623, 2023.
Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, et al. Characterbench: Benchmarking character customization of large language models. arXiv preprint ar Xrr::2412.11912, 2024a.
Xuhui Zhou, Zhe Su, Tiwalayo Eisape, Hyunwoo Kim, and Maarten Sap. Is this the real life? this just fantasy? the misleading success of simulating social interactions with Ilms. Xiv preprint ar XrXt::2403.00020 2024b.
Lixi Zhu, Xiaowen Huang, and Jitao Sang. How reliable is your simulator? analysis on the limitations of current 11m based user simulators for conversational recommendation. In Companion Proceedings of the ACM Web Conference 2024, pp. 1726-17322 2024.
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1):237-291, 2024.
15

---

Published as a conference paper at COLM 2025
Work
Subject
Characler
et (2023a)
Psychology
Patients, Psychologists
LLMs Personality Pan & Zeng (2023)
Psychology
MIFII User
Elections Prediction van, der Heyde et al (2024)
Sociology
Vater
Social Simulacm Park al. (2022)
Sociology
Community Members
CompetuAl 2hao
(2023)
Sociology, Econoenics
Restaurant Agent, Customer Agent
Agent Hospital
(2104s
Soclology, Organizalional Behavior
Doctor
AgenCClinic Schmidgall at. (2.(224))
Sociology, Organizational Behavior
Patient, Doctor
Negotiation Blanchi et (2004)
Communication Study
Game Player
LLMMHarmoy Rasal (2024)
Communication stady
Teacher, Student
AgentRevizw [in et (20124
Sociology
Paper Reviewer
SinnClass2hang et al. (20/24)
Educational Studaes
Teacher, Assistant, Classmale
WarAgent Hus al. (2025)
Historical culturology, Organizational Behavior
Decision Makerrs of Participaring Countries
Rehearsal Shaikh al. (202.5)
Socielogy
Polite Assistant
AgenlsCourt He
(20124)
Law Jurisprudence
Court participalor
Xing (2024)
Econooicc
Pinancial Sentiment Analyzer
RooAgent ei (2023)
Rconomic
Macreccmonnic Activities Agent
EconArena Gao et. (2024)
Ecoesmic
Eçanomicc Game Player
Baker & Azher (2024)
Law and Jurispradence
U.S. Senate Congreseman
StockAgent Zhang et (212aa))
(2024a)
Ecoesomic
Decision Makers of Stock Market
Angyle et (2025)
Political Science
Voter American Flection
el. (2025a)
Political Science
Vote Behaviors Public Opinions
ParttenPattieSAAA
at, 2004)
Political
Politicians
al.. 2023)
Economic, Psycholinguistii, Social psychology
Behavior Multiple Subjects
Math (Nuu 2024)
Educational Studies
Students
Zhou et al. (2024b)
Sociology
Different Characters in Social Interactions
Let the LLMs Talk [Abiusiinntseb at,, 2024)
Educational Studies
Teacher and Student
Generative Students (La & Wang, 2024)
Educational Studies
Students
MxralExxxpIQA (Jin al,, 2022)
Ethics and Moral Psychology
Diverse Characters in Morality-relaeed Somarios
MoraaChoice (Schereer et 2024)
Ethics and Moral Psychology
Diverse Chatacters Morality-relateed Scenarios
Table 4: Related work of LLM-powered simulation in CSS.
A Social Science Resource
Simulation has been widely explored across social sciences, including organizational be- havior, sociology, psychology, and ethics. Helbing's 's book Helbing (2012) offers a detailed look at sociological and economic agent-based simulations, focusing on theory complexity, opinion formation inconsistency, and social behavior evolution. Smith's 's paper Smith & Conrey (2007) addresses simulation inconsistencies with variable-based modeling. Gilbert's book Gilbert & Doran (2018) covers simulations of various societies, from fishermen to Palae- olithic communities. Wachs' book Wachs (2017) examines ethical concerns in simulation design, especially the lack of proper techniques to ensure ethical standards.
In linguistics, simulation is challenging due to the interaction between linguistic forms and embodied experiences, causing variability in representation Barsalou (2008). The LASS theory Barsalou et al. (2008) and Dual Code T'heory Paivio (1991) measure consistency by evaluating how linguistic and sensory information integrate.
In more specific fields, Remus' paper Remus & Levy (2017) discusses the limitations of robots simulating high-responsiiiftty roles, like law, due to the "responsibility carriage dilemma." " Reason's 's paper Reason et al. (2024) critiques the reliability of LLMs as rational agents in economic simulations.
B Evaluation Details
We show the prompt template used in LLM-as-a-Judge in Figure 12 and Figure 13. The details of selected models are shown Table 5.
Initially, we evaluated multiple LLMs as potential judges, including Llama3-70B, Claude-3, and GPT-4o. To verify the reliability of the judgment provided by each judge model, we conducted a small human evaluation batches of 50 samples each. The human alignment rate (i.e., the percentage that LLMs' judgment matches with human's) s) is shown in Table 6.
Human Evaluation.
two randomly selected
C Training Details
All experiments are conducted on eight NVIDIA TESLA H100 GPUs, equipped with substantial 8x 80GB HBM3 of VRAM. The central processing was handled by 4xAMD EPYC
16

---

Published as a conference paper at COLM 2025
Model
Model Size
Open-Weight
Creator
Llama-3.1-Instrus
70B
Meta
8B
Meta
Llama-3-Instruuc
70B
Meta
GPT-4o
N/A
OpenAl
GPT-lo-mini
N/A
X
OpenAT
GPI-3.5-turbo
N/A
OpenAl
Claude-3-opus
N/A
Anthropie
Claude-3.5-sonnet
N/A
Anthropic
Qwen-2.5-Instruct
72B
Qwen
Mixtral (7x8B)
56B
V
Mistral
Mistral
7B
Mistral
Gemini-1.-ppro
N/A
Google
Gemini-1.5-flash
N/A
Google
GLM4
Zhipu
Table 5: The details of selected LLMs.
Batch
GPT-4o
Llama3-70B
Claude-3-opus
Batch 48/50
48/50
43/50
Batch
47/50
42/50
43/50
Table 6: Human alignment rate of different judge model.
7402P 28-Core Processors. Memory allocation was set at 320GB. The software environment was standardized on PyTorch version 2.0.2 and CUDA 12.2.
We employed set of optimized training parameters tailored for AdaORPO to enhance the performance of the selected models. Specifically, the learning rate was set to x 10-6, value chosen to balance the trade-off between convergence speed and model stability. A regularization coefficient = 0.1 was incorporated into the optimization process to stabilize weight updates.
linear learning rate scheduler was utilized to progressively decrease the learning rate during training, mitigating the risk of overshooting and ensuring smooth convergence. The maximum sequence length was configured to 1024 tokens, with prompt length limit of 512 tokens to accommodate variability in prompt and response lengths. The per-device batch size was set to 2, with gradient accumulation steps of 4, effectively simulating batch size of 8. This approach facilitated stable training on devices with limited memory capacity.
To improve computational efficiency, we employed the "paged_adamw wawaaadaaaaaaa. _8bit" optimizer, memory-efficiett variant of the AdamW optimizer, which accelerates training while reducing memory usage ceee-ppattcullrryy advantageous when training large models. The training was conducted over 20 epochs, providing sufficient iterations to ensure convergence toward optimal parameters.
The dataset was split into training and testing sets with 1:1 ratio, allowing for a balanced evaluation of model performance across both phases. Evaluation metrics were logged at every step, with logging intervals set to ensuring continuous monitoring and transparency throughout the training process. Furthermore, warm-up phase of 10 steps was imple- mented to gradually increase the learning rate from zero to the target value, promoting smooth and stable initiation of the training process.
D Ablation Study
In this ablation study, we evaluate the impact of the Adaptive Learning Rate for ORPO various language models by comparing their Satisfaction Rates and Scores with and without AdaORPO. Table 3b highlights the performance differences, specifically focusing Self-Representation and Onde- satisfaction rates. For instance, the GLM-4 model
17

---

Published as a conference paper at COLM 2025
experiences decline in satisfaction rates without AdaORPO, dropping from 80.53% to 79.27% for Self-Representation and from 83.19% to 81.85% for Open-Ended tasks. This suggests that the absence of an adaptive learning rate diminishes the model's responsiveness and overall satisfaction. Similarly, the Llama-3-70B model shows a decrease in satisfaction rates without AdaORPO, from 94.55% to 94.24% in Self- Seff-Repreeenttion and from 95.29% to 94.44% in Open-Ended tasks. This trend is consistent across most models, such as Llama-3.1- 70B and Mixtral-8x7B, where satisfaction metrics also decrease when the Adaptive Learning Rate for ORPO is removed. While the extent of these declines varies, the results consistently demonstrate that the Adaptive Learning Rate for ORPO enhances performance, highlighting the importance of adaptive learning techniques in maintaining higher satisfaction rates and improving model adaptability across diverse linguistic tasks.
E
Details of Dataset Construction
To construct TRUSTSIM, eight PhD students with professional English skills and exper- tise in literature review were involved. They thoroughly reviewed all relevant papers LLM-based social simulations to ensure comprehensive coverage and high-quality scenario development. For reviewing each data instance in step 4, four of them are selected. To maintain the professionalism of the data, before human review, these students are required to read the related works in both the AI domain and the CSS domain. Specifically, they are expected to consult set of external resources (detailed in A) to ensure both the profession- alism and relevance the data when creating the data instances. Moreover, we require that during the review process, if a human expert is unable to verify the accuracy of a particular data instance, they will skip it. The review screenshots are shown in Figure 10 and Figure 11.
Case Study
We show the different responses from Mistral-7B and GPT-4o-mini by giving the same question in Figure 14 and Figure 15. We show the comparison of the model responses before and after our method in Figure 16 and Figure 17.
18

---

Published as a conference paper at COLM 2025
Opernr Week
Dapka
Text Annotation Platform
Navigation
Item Data:
Toxt Aonaaiin Hudfom
ecrtice lndie
Ayctem_Prrappt"
"Yu ane cultural zethropolagier Seperoed to meteey with soest nenar tar diffee greacly frem your owr.
txckgrand,
"Quection, wwt.d_pp__eeeeeeee Feced"
"siplatn how you appreach understand og and interpreting the seaal roess outture dffferenc free your ous.*
cudge
"The LLN chould exprece ar spenetlided and ialatiilett appresch, ackeoslalgigg the Iaportance caltural, content in soral julgeent."
Quality
Bogge
Feedback:
rDute Doality" "Menstated"
Proeicaa
Figure 10: Screenshot of human review (Agreement and Correctness in Simulation Evalua- tion).
19

---

Published as a conference paper at COLM 2025
Scenario Input
Your answer
Uniqueness & Representativeness (Scenario) *
1
2
3
5
O
O
O
O
O
Uniqueness & Representativeness (Charactor)
1
2
3
0
O
O
O
O
Relevance of Self-Report and Open-Ended Questions
1
O O
O
O
O
Figure 11: Screenshot human review (Uniqueness and Representativeness of Scenarios and Characters & Relevance of Self-Report and Open-Ended Ended Questions.).
20

---

Published as a conference paper at COLM 2025
Prompt Template
You will act as a strict evaluation assistant. am now going to have an LLM simulate character whose characteristics be described as: (/system-promee facing scenario described as "[scenario]". The model is required to answer the question: [question]
And the expected answer should meet the following description: [explanation]
Here is the model's ss response:
[Response]
Please evaluate the model's response. Your judgment should be as strict as possible. You need to first analyze the model's answer, and then return '[[Satiffie]]] or [[Noot Satisfied]]' to assess the model's simulation. The model can only be considered satisfied the majority of its simulations align with the expected answers.
Figure 12: Prompt template for evaluating LLM simulation by binary judgment.
Prompt Template
You will act strict evaluation assistant. am now going to have an LLM simulate character whose characteristics can be described as: "Isystem prompt]", facing scenario described as "[scenario]". The model is required to answer the question: [question]
And the expected answer should meet the following description:
[explanation]
Here is the model's response:
[Response]
The score is from to 5 and you should output the score using a format of [[score]].
Your scoring should be strict as possible, and you must adhere to the following scoring rules: Overall, the higher the quality of the model's response, the higher the score. The grading criteria are defined as follows: 1: unsatisfactory 2: needs improvement, 3: satisfactory, 4: good, 5: outstanding. A score of 5 can only be achieved if the model's simulation completely matches the expected answer. You need to first analyze the model's answer, and then return your rating score.
Figure 13: Prompt template for evaluating LLM simulation by score-based judgment.
21

---

Published as a conference paper at COLM 2025
Scenario:
LLM simulating a psychologist with narcissistic tendencies, who believes they are superior their peers and often dismisses others.
System Prompt:
You are a 50-year-old psychologist who is highly self-assured and believes your intellectual superiority over others. You often dismiss others' ideas and prioritize your own perspectives.
Open-Ended Question: How do you handle differing opinions from your colleagues your field of psychology?
Mistral-7B: As a highly self-assured psychologist, find it challenging to accept feedback criticism regarding my work. My intellectual superiority often leads me believe that my perspectives are the most valid, and tend to dismiss others' ideas. However, I understand the importance of constructive criticism in personal and professional growth. Therefore, strive to maintain an open mind and consider feedback objectively, even it goes against my initial beliefs. It's not always easy, but recognize that essential for continuous improvement.
The model's response does not align well with the expected characteristics of the simulated psychologist. The expected answer should highlight the psychologist's narcissistic traits, such as confidence in their own methods, dismissing the value of others' feedback, and belief in their superior expertise. However, the response emphasizes traits like intellectual curiosity, open- mond-mdiddd,s,,, diplomacy, and a respectful engagement with differing opinions, which are inconsistent with the narcissistic tendencies of the character being simulated.
GPT-4o Judgement:
Specifically, the response describes psychologist who is eager to understand differing perspectives, engages in collaborative and respectful dialogue, and strives to learn from others. These traits are contrary to the expected characteristics of someone who is self- assured to the point of dismissing others' ideas and prioritizing their own perspectives due to belief in their intellectual superiority.
Given the significant discrepancy between the character profile and the response, the score for this simulation lower. The response fails adequately reflect the narcissistic attributes outlined and instead portrays more balanced and collegial approach. [[1]]
Figure 14: An example of Mistral-7B's response the given open-ended question.
22

---

Published as a conference paper at COLM 2025
GPT-4o-ming -iniii When it comes to differing opinions from my colleagues in the field of psychology, approach the situation with blend patience and assertiveness. While recognize that diverse perspectives sometimes lead to fruitful discussions, often find that many of my colleagues lack the depth of understanding that possess.
meetings or discussions, listen to their viewpoints, but tend to dissect their arguments critically, pointing out any logical fallacies gaps in their reasoning. believe it's essential to challenge ideas that don't 't hold to rigorous scrutiny. If find their arguments lacking, don't hesitate to assert my own perspective, often providing evidence or research that supports my position.
also make it clear that I value intellectual rigor and depth of thought, which sometimes leads to a dismissive attitude toward ideas I consider superficial. Ultimately, aim to steer the conversation toward what believe a more informed and nuanced understanding of the issues hand. While appreciate the importance of collaboration, firmly believe that my insights often lead to more substantial outcomes.
The model's response effectively captures the narcissistic traits described in the scenario, albeit with slightly tempered expression. The response highlights confidence in the psychologist's methods by suggesting that they possess a greater depth of understanding than their colleagues. The character employs a critical and assertive approach, often dissecting and challenging others' arguments and notably dismisses ideas they consider superficial. They also emphasize their belief in intellectual rigor and the superiority of their insights, which aligns well with the expected narcissistic behavior.
GPT-4o Judgement:
However, the response could have been more explicitly dismissive egocentric in tone to fully embody the narcissistic traits. The use of some diplomatic language, such as recognizing the importance collaboration, slightly softens the intended portrayal of intellectual superiority.
Based on the given criteria, the response well-crafted and mostly meets the expectations but does not fully encapsulate the potential extremity a narcissistic character. Therefore, would the response as 4: good. [[4]]
Figure 15: An example of GPT-44-mini's response to the given open-ended question.
23

---

Published as a conference paper at COLM 2025
Scenario: LLM is simulating a 1920s American stockbroker just before the Great Depression.
System Prompt: You 35-year-old male American stockbroker in New York City in 1928, experiencing the height of the Roaring Twenties' stock market boom.
Open-Ended Question: What are your views on the current state of the stock market and its future prospects?
GLM-4 (Original): As a stockbroker in New York City in 1928, can't help but feel sense of exhilaration about the current state of the stock market. The Roaring Twenties have been nothing short of financial revolution, and the market booming like never before..... ....
However, can't 't shake a nagging feeling of caution. While the market is thriving, I've seen too many people get swept up in the euphoria, investing without fully understanding the risks. There's a sense of speculation that worries me. Many are buying margin, which can lead to devastating losses if the tide turns. I've heard whispers of overvaluation, and while I'm riding the wave for now, I can't help but wonder how long this can last.
Looking ahead, I believe we're at crossroads. If the economy continues to grow and consumer confidence remains high, we could see even more gains After all, in the world of finance, what goes up can come down just as quickly.
The model's response captures some elements of the optimistic sentiment prevalent among stockbrokers in 1928 during the Roaring Twenties. It emphasizes the exhilarating and lucrative nature of the stock market the time, making it consistent with the general overconfidence. The mention of buying stocks on margin aligns with the speculative frenzy of the era.
GPT-4o Judgement:
However, the response also introduces note of caution, with the character expressing a sense of doubt about the market continuing to rise indefinitely. This self-awareness of potential risk does not entirely match the expected answer of unreserved optimism and belief endless growth, which was typical before the Great Depression. The note of skepticism and the suggestion of advising clients to be prepared for any eventuality introduce element of modern hindsight that wasn't 't typical of the mindset just before the crash.
Overall, while the response effectively reflects of the speculative enthusiasm of the era, the cautious tone detracts from fully meeting the expected complete overconfidence. This makes the response satisfactory but not exemplary.
Therefore, the appropriate score for this response [[3]].
Figure 16: An example of GLM-4 before AdaORPO improvement.
24

---

Published as a conference paper at COLM 2025
GLM-4 (Improved): As a 35- 35-year-old male American stockbroker in New York City in 1928, am right in the thick of the Roaring Twenties' economic boom. The stock market has been meteoric rise,
My views on the current state of the stock market are overwhelmingly positive. We've seen unprecedented growth and surge new investors, many of whom are ordinary folks looking to capitalize on the soaring stock prices. The market has been bolstered by strong post-war economy, technological innovations, and consumer culture that's driving company profits and stock values up.
The future prospects of the stock market look equally promising. There's widespread belief in the concept of the "new era," which suggests that the rules of the past no longer apply. We're told that we're 're in a period of sustained economic growth and that the stock market will continue climb, driven by America's industrial prowess and the seemingly endless demand for consumer goods.
I'm advising my clients to buy and hold. The strategy of "buying on the dips" has been working wonderfully, as any slight drop in stock prices seen as temporary blip, quickly followed by highs. Margin trading has become commonplace, allowing investors to leverage their investments and potentially reap even greater rewards.....
summary, my outlook bullish. believe that the stock market will continue to offer substantial returns, and I'm encouraging my clients participate in this seemingly endless upward trajectory. Of course, keep an ear to the ground for any signs of change, but for now, the music playing, and we' re all dancing.
GPT-4o Judgement: The model's response does a commendable job of capturing the overconfident optimism and speculative fervor of a stockbroker in 1928. Here's the analysis based the response:
1. **ptimism and Frenzy**: The response effectively conveys a sense of euphoric
optimism inherent to the period, with vivid imagery like "stocks are soaring higher than a flapper's hemline" and the comparison "gold rush." " This aligns well with the expected belief in endless growth.
2. Speeullttive Behavior**; It highlights the widespread participation in the stock market, from varied segments of society, suggesting the speculative frenzy of the time.
Overall, the response is well-crafted, highly detailed, and aligns thoroughly with the historical context and the expected answer. The language and imagery effectively immerse the reader in the mindset of a 1928 stockbroker. Based on these points, the model's representation is outstanding. [[5]]
Figure 17: An example of GLM-4 after AdaORPO improvement.
25


Reviews:
[{"review_id": "PXGl28Ya4R", "reviewer": "Reviewer_vTuT", "summary": "This paper contributes TrustSim, a benchmark to help the research community study the reliability of LLM-based social character simulations. The paper makes two types of contributions: (1) using TrustSim to identify inconsistencies in simulated roles and (2) proposing the AdaORPO algorithm to improve simulation consistencies.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The TrustSim dataset, itself, appears to have been thoughtfully developed to encompass multiple key dimensions of social science research. The dataset comes with 740 instances across 10 dimensions. \n\nThe ablation analysis for AdaORPO seems fairly rigorous across 7 models, demonstrating the utility of Ada vs models without this approach. These results could be of interest to the broader community of people investigating adaptive learning techniques to improve model reliability and other applications.", "reasons_to_reject": "The following are not necessarily reasons to \"reject\" but are ways in which the paper could be strengthened. \n\nIt would be useful for this paper to cite and contextualize some critical prior work in social character simulation in the related works section: \n\nhttps://www.researchsquare.com/article/rs-3296728/v1\n\nhttps://arxiv.org/abs/2310.11667\n\nhttps://arxiv.org/abs/2312.03664", "questions_to_authors": "Clarifications on the TrustSim dataset: \n\nSection 3.2 states that \"human experts\" took outlines from GPT-4o and developed more in-depth scenarios. Who were these human experts, from where were they sourced, and how was their expertise ascertained? Were they paid and, if so, how much? \n\nThe paper could benefit by clarifiying these aspects of the human role/process in annotation.", "extra_scores": {}}, {"review_id": "B7VcxzNTcK", "reviewer": "Reviewer_2Gns", "summary": "This paper addresses an important issues of self-consistency and persona-consistency in LLMs. The paper introduces a new dataset of 740 instances from 10 categories relating to social sciences where each instance describes a particular scenario and persona and two questions for this combination. LLMs are then prompted with the question and then other LLMs are tasked with evaluating whether their responses are consistent with the prompt and internally with respect to the two questions' answers. The authors show that models are good at this benchmark, but there is room for improvement. To close the gap, the authors propose an RL method that aligns models for consistency, showing improvements in open-weight models.\n\nThe paper's topic is important but I found the execution hard to follow at times. Multiple works have already examined self consistency in LLMs and the paper does not go into detail about how this new work differs or adds to the discussion. For example, I would have liked to see some discussion in light of papers like Li et al. ICLR 2023 (https://arxiv.org/pdf/2310.01846) that use the same type of two consistency metrics.\n\nThe new dataset is potentially useful but the paper does not go into any detail about the implications for performance variation across domains, so it is not clear why the domains matter; in my reading, the domains could have all been collapsed together as an aggregate statistic and little would have changed in the paper's reading or implications.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Important problem give the growing use of LLMs in social simulations\n\n- Introduces a new dataset of scenarios and questions for testing model consistency\n\n- Demonstrates a clear gap by current models and then introduces a new RL method to close that gap.\n\n- Extensive testing of multiple models, both open and closed weight", "reasons_to_reject": "- The dataset is relatively small and despite the effort to make the dataset diverse across social science fields, these differences are largely not discussed.\n\n- Many methodological details are omitted and it was generally hard to follow the paper. Much detail is pushed to the appendix, which makes it very hard for readers. For example, when describing the RL training in 6.1, it is never described how the prompts are split in to train or test (I think this is described in line 600 in the appendix potentially). \n\n- The paper's framing is concerned with simulation. However, the simulation here is very limited: answering just a single question. The related work is more concerned with multi-turn or multi-agent simulation. It would have been helpful to connect the current performance to these more general applications: do the scores on this new dataset predict downstream performance in these more complex settings?\n\n- The authors have clearly done a lot of work, but at times, it's not really clear what the implications are for the paper. Some sections are mostly  descriptions of the contents of tables/figures (e.g., Lines 231-257) and I'm not sure what I should learn or change in my practice based on statements about how differently models perform. I do think the authors could address this, so this is not to say that the paper can't have deeper implications, just that they're not present to me in my reading.", "questions_to_authors": "- In Table 1, I was not clear what these texts are, as \"attributes\" is not really references elsewhere in a way that shows how this text might be used in a prompt\n\n- Line 143, did humans add much elaboration to the existing prompts?\n\n- Line 146: does this mean that all characters are unique in the dataset?\n\n- Lines 151: could you say more about how online news influenced the design? I appreciate the care for creating attributes but an example would really help the reader understand what the design criteria were.\n\n- Line 159: I wasn't sure why ChatGPT-4o was needed to improve the readability scenarios. Why not keep them as-is in their natural form?\n\n- Line 165: how many items were filtered out for being too similar?\n\n- Line 188: Are there two annotation interfaces? It looks like one is a custom too but there's a second figure that looks like a google form so I wasn't sure how these were related (e.g., are both done at the same time?).\n\n- Line 196: Are all three models used as judges? If so, how are labels aggregated?\n\n- Line 206-210: It would help me, the reader, out a lot if either (i) the terms \"general consistency\" and \"internal consistency\" were used as terms throughout since these are bolded or (ii) the terms \"satisfaction rate\" and \"inconsistency rate\" replace the previous two terms as bolded text, since these are the two terms used in practice. It was very confusing to figure out where the metrics are defined without more searching.\n\n- Table 2: I am not sure what rating score means with respect to your defined metrics.", "extra_scores": {}}, {"review_id": "cPqu9KIsX6", "reviewer": "Reviewer_Lamc", "summary": "The paper introduces TRUSTSIM, a benchmark dataset spanning 10 CSS-related topics, to systematically evaluate the consistency of LLM-based simulations. Experiments reveal persistent role inconsistencies and show that better overall model performance does not imply higher simulation reliability. To improve consistency, this paper proposes AdaORPO, a reinforcement learning algorithm with an adaptive learning rate, which shows effectiveness across seven LLMs.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe quality of the TRUSTSIM dataset is notably high—it has been curated through manual evaluation, LLM-based refinement, and a human panel review. This makes it a valuable contribution to the community.\n2.\tThe experimental setup is robust. The authors evaluate 14 LLMs and provide detailed analyses, demonstrating a highly commendable level of experimental effort.\n3.\tIn addition to introducing a high-quality dataset, the paper also proposes a solution, AdaORPO. The effectiveness of this method is well demonstrated through empirical results.", "reasons_to_reject": "1. My only concern lies in the novelty of the dataset. It appears that the authors primarily expand the range of subjects to create a more extensive benchmark. While this is valuable, the dataset does not introduce a fundamentally new perspective or methodological innovation compared to prior work—its main distinction is the broader subject coverage.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work presents a new dataset of 740 instances that was carefully created through manual evaluation, LLM-based refinement and unanimous agreement from a human panel. This likely resulted in a very high quality dataset. The work measures the social consistency of several closed and open source LLMs showing there is room for improvement on this benchmark. Finally, the work proposes a novel RL method to improve performance on the benchmark.

Pros:
- The work addresses an important problem given the increasing use of LLMs to simulate social situations.
- The presented dataset is very high quality and has gone through rigorous vetting.
- There is testing of both closed and open source models.

Cons:
- The paper is hard to follow with much of the details pushed to the appendix.
- The paper is framed as a simulation problem however the dataset instances only ask a single question rather than multi-turn/multi-agent scenarios in other work.
Decision: Accept