Social Agents: Collective Intelligence Improves LLM Predictions
Authors: Aanisha Bhattacharyya
Abhilekh Borah
Yaman Kumar Singla
Rajiv Ratn Shah
Changyou Chen
Balaji Krishnamurthy
Venue: ICLR 2026 Poster (2026)
Keywords: wisdom of crowds
LLM
multiagent systems
Primary area: foundation or frontier models, including LLMs

Abstract: In human society, collective decision making has often outperformed the judgment of individuals. Classic examples range from estimating livestock weights to predicting elections and financial markets, where averaging many independent guesses often yields results more accurate than experts. These successes arise because groups bring together diverse perspectives, independent voices, and distributed knowledge, combining them in ways that cancel individual biases. This principle, known as the Wisdom of Crowds, underpins practices in forecasting, marketing, and preference modeling. Large Language Models (LLMs), however, typically produce a single definitive answer. While effective in many settings, this uniformity overlooks the diversity of human judgments shaping responses to ads, videos, and webpages. Inspired by how societies benefit from diverse opinions, we ask whether LLM predictions can be improved by simulating not one answer but many. We introduce Social Agents, a multi-agent framework that instantiates a synthetic society of human-like personas with diverse demographic (e.g., age, gender) and psychographic (e.g., values, interests) attributes. Each persona independently appraises a stimulus such as an advertisement, video, or webpage, offering both a quantitative score (e.g., click-through likelihood, recall score, likability) and a qualitative rationale. Aggregating these opinions produces a distribution of preferences that more closely mirrors real human crowds. Across eleven behavioral prediction tasks, Social Agents outperforms single-LLM baselines by up to 67.45% on simple judgments (e.g. webpage likability) and 9.88% on complex interpretive reasoning (e.g. video memorability). Social Agents’ individual persona predictions also align with human judgments, reaching Pearson correlations up to 0.71. These results position computational crowd simulation as a scalable, interpretable tool for improving behavioral prediction and supporting societal decision making.

Published as a conference paper ICLR 2026
SOCIAL AGENTS: COLLECTIVE INTELLIGENCE IMPROVES
LLM PREDICTIONS
Aanisha Bhattacharyya"
E IIID
Abhilekh Borah
Yaman Kumar Singla"
Rajiv Ratn Shah
IIID
Changyou Chen
Balaji Krishnamurthy
Adobe Media and Data Science Research (MDSR)
IIDD Indraprastha Institute of Information Technology Delhi
S State University of New York at Buffalo
behavior-in-the
ABSTRACT
In human society, collective decision making has often outperformed the judgment of indi- viduals. Classic examples range from estimating livestock weights predicting elections and financial markets, where averaging many independent guesses often yields results more accurate than those of experts. These successes arise because groups bring together diverse perspectives, independent voices, and distributed knowledge, combining them in ways that cancel individual biases. This principle, known as the Wisdom of Crowds, un- derpins practices in forecasting, marketing, and preference modeling. Large Language Models (LLMs), however, typically produce a single definitive answer. While effective in many settings, this uniformity overlooks the diversity of human judgments shaping re- sponses to ads, videos, and webpages. Inspired by how societies benefit from diverse opinions, we ask whether LLM predictions can be improved by simulating not an- swer but many. We introduce Social Agents, a multi-agent framework that instantiates a synthetic society of human-like personas with diverse demographic (e.g., age, gender) and psychographic (e.g., values, interests) attributes. Each persona independently appraises a stimulus such as an advertisement, video, or webpage, offering both quantitative score (e.g., tlick-through likelihood, recall score, likability) and a qualitative rationale. Ag- gregating these opinions produces distribution of preferences that more closely mirrors real human crowds. Across eleven behavioral prediction tasks, Social Agents outperforms single-LLM baselines by up to 67.45 37.45% on simple judgments (e.g. webpage likability) and 9.88% complex interpretive reasoning (e.g. video memorability). Social Agents' individual persona predictions also align with human judgments, reaching Pearson corre- lations up 0.71. These results position computational crowd simulation scalable, interpretable tool for improving behavioral prediction and supporting societal decision making 8
INTRODUCTION
"The many, of whom none a good man, may yet, when joined together, be better than those few." Aristotle
Equal Contribution.

---

Published as a conference paper at ICLR 2026
Rirose the wewtetsgee 5.8 Familaar revopppper dioe spout clear headines and mony links mase trowsing easy whecet Bualty
caoe the antpugg 2.5 The styirgisca atfizuni, with a paleto, small imagens
Tirrr informatiion dense winh helpttll canggry soto and scannahle hood iree, but worta oficcenly only once you Ieam
What the Five Personas Ratings (on the lnny Show Us About Collectivn Judggmente with Social
space
Agants??
DAILY
36 year old teacher
Divendity of opinion: Diferent
personas rase the wetpage high low tasaed bicader
22 a freb clstere and doodd with wook road honucty that maken nowt. at's glanos
74cysur-old
soedal
preferences,
19-your-lle old fasition anthusiant
freed boss fast,
Independence: Each
the bung siiuuus.
metquega indrodtes being inflacnced by other persenas. Decentrakzztion: Each parsonsa diascs on own everyday cornec
sce, and week thear ciess draan
21 college student
rite the webpagee 3.86 Based the
Social Agents
18-your-oti
jatge the eetpuege wiiting local knowlddge without rely only on cammin glotel viens.
everyene' pl rating, ove al loots cluttored
What would this webpauge scale 1-107
and datted. Whil is information dense and
provtdes (sada faxt, the thssigs lands bland in
Aggregation: Social Agerns combine the persona ratinge and ther neloradie the welqaage overal verder
Iceass A larrilae tar structure arw with flaslty contrend.
scannable sectioon, and more moderr
vinunds soruld improwo for wantyonn
Figure I: Social Agents are grounded in the principle of the Wisdom of Crowds, the idea that collective judgments can outperform individual ones when based on the four key ingredients: (i) Diversity of opinion, (ii) Independence. (iii) Decentralization, and (iv) Aggregation (Surowiecki) 2004). As shown in the figure, to capture diversity of social perspectives in wehpage likability judgments, we elicit evaluations from five personas varied across age (to reflect generational viewpoints), profession (to represent occupational contexts), and social preferences (to capture differences in values and tastes). Each persona offers independent ratings and rationales, ensuring varied perspoctives, which are then aggregated into one collective verdict.
The idea that aggregated opinions can surpass the accuracy of single expert has fascinated thinkers since antiquity. Aristotle captured this insight in Politics (350 BCE) (Aristotle) (200)). observing that ordinary people, taken together, can judge music, poetry, and politics better than trained crities, since each notices a different aspect. He emphasized that diversity of skills, experiences, and insights enhances the quality of collective judgment and decision-making This early recognition laid the groundwork for what modern scholars now call the wisdom of crowds (Schwartzberg 2016).
A classic empirical demonstration came from Galton's 1906 analysis of livestock fair, where nearly eight hundred participants estimated the weight of an ox. Although individual guesses varied widely, the me- dian was within one percent of the true weight, outperforming even the best single estimate and showing robustness to outliers (Galton, 1907). Building on this, /Srrowickki (204) popularized the idea that group decision- maainn can outperform individuals when four conditions hold (Figure D: diversity of perspec- tives, independence of judgment, decentralization of knowledge, and proper aggregation. Over the past two decades, these principles have been applied in practical systems. One example Best Buy's 2005 internal prediction market, Tag Trade, where employees used virtual currency to bet on outcomes such as holiday gift-card sales. The market's forecast proved accurate, missing actual sales by just 0.6% (Haas et al. 2016). Modern platforms like Polymarket apply the same principles, showing how collective estimation can surpass experts.
We have seen that the wisdom of crowds can yield remarkable accuracy across domains ranging from live- stock competitions to geopolitical forecasting. Yet these successes have relied on human participants, which imposes limitations: assembling and incentivizing large groups requires sustained effort, is costly to scale, and cannot easily be applied to every decision making scenario.
As large language models (LLMs) grow more capable, recent systems such as GPT-4o (Hurst et al. 2024) and DeepSeek-R1 (DeepSeek 2024) now perform complex reasoning tasks once reserved for humans, ex- celling in mathematics, code generation, and cross-modal analysis. Many are pretrained on user forum data such as Reddit and other discussion platforms (Radford et al. [001 Gooassaa & Cohen 2019; Demszky ett al.; 2020), giving them implicit grasp of how personas behave and how individuals with specific psy- chographic interests interact. For example, (Santurkar et al. 2023) demonstrated that persona prompting can steer LLMs to respond in the voice of given persona across social and apinion-rrlateed questions. Similarly,
2

---

Published as a conference paper ICLR 2026
What will be the CTR percentile of this
Parcentlle Advrilismant:55
advertisement 0-1007
Predicted CTR Percentile by Social Agents: 54
ad' aesthatic appeals strongly creative personas who jappreciats calm, elegant visuals and refined tone. resonates wel with those seeking simplicity and inspiration. However, the lack dynamic content resutts lower engagement from younger and oriented sudiences, who often look for boldness novelty creetive looil.
Serve Terger Contest for each Pereona Agent
Retrieve Top-K ads use them
fow--hoot examples to ald prediction
Aggregate Predictions via Mean
Persona Agent Factory
CTR Percentilee 66 Reason: the CTR percentle 66, elegant viruals and minimal design repeal to creatives, thouch dynamic elemenns may limit excitement...
CTR: 92 Passor:
CTR: 48 Roasen:
55+ Isrrale
10-24 tamale Rosiioo
Two grandkiss,
Tool graduate fumily
Markuling graduate
Moderator Diverse Personas
13-17 female eniiusiest Student
Student, STEM
with a laeely
anthusinet
!!
CTR: 60 Reasort
CTR Resson:
25-31 male
34-43 male
Creative
19:17 male
Markeiigg
gracustte
Tach
gradiaste
Student Arts
Simulated Society
graduste
Figure 2: Overview of the Social Agents workflow for Ad Click-Through Rate (CTR) Prediction. Given an advertisement (top-left), our framework computes its embeddings and retrieves the top-K semantically similar ads from repository of ad embeddings. These serve as few-shot examples that aid CTR prediction. Factory (bottom-left) contains personas defined by demographic attributes (e.g.. age, gender) and traits (e.g.. interests, occupation), following templates in Appendix Table 2 From this pool, the moderator selects a diverse panel of N personas and instantiates separate LLM agents for cach. Each persona agent outputs CTR percentile (0-100) with brief rationale. The right-hand side shows the moderator aggregating these predictions via mean to produce a single CTR percentile, with a collective rationale that is compared against the ground-truth CTR percentile.
Persona Agent
Tess et al., 2025) showed that persona prompting allows LLMs to serve as proxy users in psychological and social studies. Together, these studies suggest that when steered to adopt a particular persona, LLMs are capable of embodying that perspective and generating responses consistent with it.
While the wisdom of crowds reliably produces accurate judgments with human groups, scaling it in practice remains costly and logistically difficult. LLMs, however, can be steered to act as human-like participants through persona prompting, providing scalable way to operationalize the wisdom of crowds in synthetic settings. This motivates our central question: "Can the wisdom of crowds be scaled to LLMs, where each instance embodies distinct persona and their aggregated responses improve the predictive and reasoning performance of language models?"
Social Agents. We introduce Social Agents, synthetic crowd intelligence framework that operationalizes the wisdom of crowds principle with ensembles of LL.M-based human-like agents. These agents are delib- crately diverse in demographic and psychographic characteristics, ensuring a wide range of perspectives. To preserve independence and avoid groupthink, they are prompted separately. Each agent grounds its reasoning its assigned profile, providing decentralized basis for judgment. Their outputs are then aggregated into a collective outcome. In this way. Social Agents capture the four classical pillars of the wisdom of crowds: diversity, independence, decentralization, and aggregation, demonstrating that collective intelligence can improve LLM predictions. This aggregation also offers three advantages. First, error cancellation occurs when individual errors offset, bringing the aggregate closer to ground truth. Second, robusiness through di- versity arises as heterogeneous perspectives guard against systematic bias and outliers. Third, interpretable communal dynamics emerge from the distribution of rationales, revealing drivers of consensus and disagree- ment. Each agent functions simulated evaluator with unique profile, providing quantitative predictions and qualitative justifications. Aggregating these via a simple mean mitigates variance and bias, improving the accuracy and reliability of population-leve predictions.
3

---

Published as conference paper ICLR 2026
We conduct a comprehensive empirical evaluation of Social Agents across suite of challenging tasks. To ground this evaluation, we draw on Construal Level Theory (CLT) (Trope & Liberman 2010), which posits that psychological distance across temporal, spatial, social, and hypothetical dimensions systematically af- fects how people represent events. CLT suggests that judgments vary in abstraction: near tasks involve immediate, surface-level evaluations, while distant tasks require deeper semantic elaboration and abstract reasoning. For example, it is straightforward for user to judge the likability of a webpage based on per- ceptual cues such as color or layout (a low-level construal), whereas predicting whether the same webpage will be remembered weeks later demands high-level construal. engaging memory, causality, and persua- sion. Following this framework, we select 11 diverse tasks spanning the CLT spectrum, including low-level predictions such click-through rate, tweet engagement, and topic action classification; medium- level predictions such as return on ad spend; and high-level constructs such as long-term memorability, causal reasoning, persuasive intent, and emotional response.
Following the wisdom of crowds principle, where collective judgments outperform individuals and often experts, our primary comparison between Social Agents and NN-PPesona variant, where single LLM expert agent is prompted as a domain specialist under the same 5-shot setting (zero-shot for Behavioral Attribute Classification). We further compare against state-of-the-ar task-specific trained expert models. LCBM (Khandelwal et al., 2024) is used for CTR, tweet generation, and tweet engagement prediction, trained on 40,000 YouTuub videos and 168 million Twitter posts. Henry (SI et al.) 2023) addresses long- term memorability, while Lahavior-LLaVa (Singh et al. 2025), trained on 730k instruction restuuction-responee pairs, is the strongest model for behavioral attribution. For ROAS and webpage likability, we include an XGBoost baseline. Together, these baselines let ask whether collective intelligence improve LLM predictions, offering a scalable alternative to methods reliant on extensive task-specific datasets and training regimes.
2.80
Best Model Task Performance
shott Best Modell Performanc
2.00
1399
2.00
+57%
+444%
1.40-
+24%%420%
+24%
1.20-
+1.5%
-13%
+6.7%
9.7%
-15%
Tweet Engagement Prediction
Ad Click-Through Rate Prediction
Return on Ad Spend Prediction
Webpage Likability Prediction
Long-Term Video Tweet Content Generation
Video Topic Classification
Video Reason Classification
Memorability Prediction
5-shot Best Model
Social Agents (Best Model)
Model Rinetuned on Täsk Specific Data
Figure 3: Performance Comparison of Social Agents (Best Model) over 5-shot Best Model and Models Finetuned on Task-Specific Data across eight tasks. Across eight tasks. Social Agents (Best Model) consistently improve over the 5-shot Best Model and often exceed models finetuned on task-specific data, despite not being trained for those tasks, Here, Best Model refers to whichever base model achieves the strongest results, whether used within Social Agents in the 5-shot baseline. Performance of Social Agents (Best Model) and finetuned baselines is reported relative a 5-shot Best Model reference (fixed 1.00). For Models Finetumed on Task Specific Data, we use: Large Content Behavior Models (LCBM) (Khandelwal al. 2024) for Tweet Engagement Prediction, Ad Click- Through Rate Prediction, and Tweet Content Generation; a trained XGBoost model for Return on Ad Spend Prediction; Heary (SI et al. (0033) for
Long-Term Video Memorability Prediction; and Behavior-LLav.. VA (Singh et al. 2025 for Video Topic and Video Reason Classification tasks.
Empirical results demonstrate that Social Agents improves performance across all construal levels. On low- level tasks involving immediate and behavior-oriente predictions, we observe an average improvement of 42.1 1,, with the largest gain webpage likability prediction, where performance rises by 164% relative
4

---

Published as a conference paper ICLR 2026
the baseline. For the medium-level task of return on ad spend prediction, which measures future financial outcomes such as whether ads will generate revenue, Social Agents achieves 153% improvement. On high-level tasks requiring abstract reasoning, the framework yields an average gain of 12%, with long-term memorability prediction (forecasting whether users will recall an ad later) showing a 24% improvement. As shown Fig. Social Agents not only outperforms the No-Persona baseline but also surpasses task- specific expert models in most cases. It achieves 24% on tweet engagement, 44% on CTR, 21% % on ROAS, and 164% on webpage likability, while also improving content generation and behavioral attribution. The only exception is memorability, where Henry performs better, though Social Agents still provides clear gains over the baseline. Overall, these results suggest that collective intelligence offers a scalable, model-agnostic path to improving LLM predictions, pointing toward a general paradigm for enhancing both predictive and generative tasks.
To summarize, our paper makes the following three contributions:
(i) Social Agents Framework. We introduce Social Agents, a multi-agent framework that operationalizes the principle of the Wisdom of Crowds with ensembles of L.LM-based human-like persona agents. Each persona agent ensures diversity of opinion, independence. decentralization, and aggregation (Figure [[]. So- cial Agents task-agnostic: evaluated across eleven diverse tasks, it consistently outperforms baselines and trained experts, yielding a 42. %% average improvement on low-level judgments (c.g., webpage lika- bility and ad click-through rate prediction) and 24% improvement on high-level reasoning tasks such as video persuasion classification and long-term memorability prediction (see Section A.2 for results and Ap- pendix [A.3] for ablation analyses). These gains are robust across decoding temperature settings, aggregation strategies (mean vs. median) (Section A.4), indicating that improvements arise from structured persona di- versity rather than stochastic variation. We also compare it against alternative ensemble strategies, including crowds- crowds-within approaches (Section A.7) and show that structured persona heterogencity consistently yields stronger performance gains.
(ii) Model-Agnostic Framework. The benefits of Social Agents model-agnostic. Evaluated across seven proprietary and open-source vision-language backbones, it delivers consistent average improvements of 23.9% over baselines across 3 models and 11 diverse tasks, with clear gains even for smaller, lower- parameter models (Appendix I..3II.
(iii) Synthetic Data Release. We release dataset of persona-conditioned predictions, definitions, and rationales from Social Agents (e.g., ad CTR, webpage likability). capturing how diverse personas and their interactions with digital content are represented across all eleven behavioral tasks.
5

---

Published as a conference paper ICLR 2026
Probability of observing Prediction Error
(Likabllity Score Prediction E (Ha, No Su.)
No--Prronna 10 Trials GPT-43)
Human Predictions
0.35
Social Agents (GPT-4o)
0.30
Notations
Webbagg
-Fl for webpage
spacific demographic group age range paired gender)
0.25
61.5 % overlap of No-
78.4 96 overlap of Social Agents with Human Predictions
Human
0.20
Persona (Mean Trials) with Human Predictions
Hi: Mean of likabliity scores for webpage / given by all humann raters
0.15
Likability Score Prediction definitions (used on X axish:
Human Predictions: Huje humann rater its likability score precliction for webpage
0.10
No-Persona (Mean of 10 Trials, GPT-4o): M, likablity
0.05
precictionn for webosge
Social Agents (GPT-4a): Sas. likabiiity prediction
0.00
specific demographic group webpage
-10.0
-7.5
-5.0
-2.5
0.0
2.5
5.0
7.5
10.0
Prediction Error for webpage / [Likability Score Prediction E (H,,,. Ni. Sa.i) - Fii)
0.0 means the Likablitty Score Prediction equals H, for webpage
Figure Comparison of Social Agents (GPT-40) prediction errors against No-Persona (Mean of Trials, GPT- 40) on the Webpage Likability Prediction task. Social Agents (GPT-4o), which incorporate individual persona pref- erences averaged across 10 diverse personas, exhibit greater overlap with the human distribution than the No-Persona (Mean of 10 Trials, GPT-4o) baseline that simply averages 10 calls per webpage. The kernel density estimate (KDE) plot show the error distributions of Social Agents and the No-Persona (Mean of 10 Trials) haseline relative the per- webpage human mean prediction 17, Shaded areas highlight, for each method, its overlap with the human prediction distribution.
2 SETUP
this section, we describe how the Social Agents framework operates. As illustrated in Fig. 2 moderator orchestrates the process by first selecting diverse panel of persona agents from the Agent Factory. Each persona is an LLM agent instantiated with the same backbone model but conditioned on different demo- graphic factors (age, gender, location) and psychographic factors (interests, values, lifestyle). The agents are prompted independently so that there is no interaction or influence among them (Appendix [A. for prompt- ing template). When presented with the target query, an agent generates a rationale from the perspective of its assigned persona and, conditioned on this rationale, provides a quantitative prediction. The individual predictions are then aggregated using the wisdom of crowds principle to produce a final ensemble score. For interpretability, the same LLM, configured in neutral and unconditioned expert mode, synthesizes the set of rationales into single collective explanation.
2.1 THEORY
Social Agents instantiate diverse persona agents, each offering an independent perspective, and aggregate their outputs. Much like the wisdom of crowds, this design captures subjective variability while mitigating bias through ensemble averaging, producing predictions that generalize across domains and approximate population-level responses. Since foundational models are pretrained on diverse human corpora, including social forums like Reddit, they inherently capture a wide range of perspectives (Hu & Collier 2024 Radford jet al. 2019; Gokaslan & Cohen, 2019). By conditioning the same backbone model on different personas, Social Agents samples this distribution of viewpoints rather than relying on single static prompt, exploiting the model's latent space to simulate psychologically grounded and socially plausible simulacra of human responses.
To illustrate, consider predicting the click-through rate (CTR) percentile for the advertisement Fig. 2 The ad's calm, elegant visuals may resonate with creative professionals but seem less appealing younger, novelty-secking users. Social Agents evaluates the ad through multiple personas: for example, a 34-45 year old female marketing graduate with family predicts 66, 25-34 year old male marketing graduate 52, 34- 35 year old tech professional 60, an 18-24 year old fashion-conscions female 42, and a 13-17 year old male 66. While these judgments differ, averaging tempers extremes and yields an ensemble score of 54, close to
6

---

the ground truth of 51. The aggregation is defined as the mean of persona agent predictions, S / Σn 1 Si Published as a conference paper ICLR 2026
where 9; is the score from persona and is the collective estimate. Unlike classical ensembles that assumee independent judgments, persona conditioning induces systematic variation: each agent draws from a distinct yet related distribution, while the shared backbone introduces correlations across agents. This structure mirrors human crowds, where collective intelligence arises from combining diverse perspectives rather than relying on brute-force repetition of the same model call.
To illustrate this distinction, we compare Social Agents to a baseline, No-Persona (Mean of 10 Trials), which averages predictions from ten repeated calls of the same LLM per webpage or ad. This setup mimics a traditional Law of Large Numbers [Bernoulli (713) setting: variation comes only from randomness in model sampling, not from systematically different perspectives. As shown in Fig. 4 on the Webpage Likability Prediction task, No-Persona (Mean of 10 Trials) produces a wide distribution of signed errors with only 61.5% KDE overlap with human predictions, whereas Social Agents achieve tighter error distribution and a higher overlap of 78.4%. The reason is that persona conditioning induces structured variation: different demographic and psychographic profiles yield distinct but complementary distributions, and aggregation cancels bias while preserving diversity. In contrast, repeated calls without personas merely reduce variance and miss the meaningful social heterogeneity that underpins collective intelligence.
Since we operate under small cost constraints due to a limited budget, we restrict N to small values, making it appropriate to compare Social Agents with the No-Persona (Mean of 10 Trials) baseline under the same conditions. As shown in the ablation study in Appendix A.3.3 both approaches plateau after about 10-20 calls per datum, indicating diminishing returns beyond this range. Thus, restricting experiments to small N (10 as default in our experiments) offers fair basis for comparison across tasks.
2.2 TASKS & DATASETS
To evaluate Social Agents across diverse cognitive domains, we adopt suite of tasks grounded in Construal Level Theory (CLT) (Trope & Liberman 2010), which posits that the level of construal, or the degree to which thinking is abstract versus concrete, varies systematically with psychological distance across tempo- ral, spatial, and social dimensions. For example, it is easier for a person to rate the likability of a webpage based on its aesthetic appeal (a near, surface-level judgment) than predict whether they will remember that webpage weeks later, which requires deeper semantic elaboration and abstract processing (a more dis- lant, high-level construal). Building on these insights from CLT, we operationalize Social Agents through eleven tasks spanning low-, medium-, and high-level construals, capturing how persona agents perceive, engage with, and retain digital content. Full formulations and task-specific evaluation rules are provided in Appendix 1.2
Low-Construal Tasks. These tasks involve immediate, surface-level, or perceptual judgments.
(1) Tweet Engagement Prediction. Persona agents predict whether tweet will receive High or Low likes, with outputs aggregated by majority vote (ties re-run). We use the LCBM dataset (Khandelwal al. 2024). evaluating on 2,339 test tweets. Performance is measured by accuracy, the fraction of correct High/Low predictions.
(2) Ad Click- Click-Through Rate (CTR) Prediction. Agents estimate the percentile position (0~100) of an ad campaign's 's CTR relative to other campaigns. CTR captures immediate behavioral outcomes, aligning with low-level construal. Evaluation uses three families of metrics: k-way accuracy, which bins both prediction and ground truth into & widhh bins (c.g.. = or 10) and checks for exact matches; PE@, the proportion of predictions whose error lies within K% of the ground-truth percentile; and error metrics, including mean absolute percentage error (MAPE), mean percentage error (MPE), and root mean squared error (RMSE).
7

---

Published as conference paper ICLR 2026
(3) Webpage Likability Prediction. Agents assess webpage aesthetics using the Web Aesthetics dataset (Reinecke & Gajos 2014) of 398 screenshots. Evaluation combines binary accuracy (thresholded at 5), error metrics (MAPE, MPE, RMSE), and Pearson correlation (r) with human ratings.
(4) Behavioral Attribute Classification (Topic, Action). Using the Image and Video Ads dataset (Hussain jet al. 2017), persona agents classify ads by semantic topic or by identifying explicit actions (e-g.. "Buy now") ). Both tasks require surface content recognition and observable cues, consistent with low construal. Evaluation is based on accuracy, computed as exact match with ground-truth labels.
Contsrull Task. These tasks involve future-oriented but still concrete outcomes.
(1) Return on Ad Spend (ROAS) Prediction. Agents predict the percentile rank of ROAS (revenue-to- spend ratio) for ad campaigns from dataset of Fortune 500 campaigns (3,422 creative industry and 410 real estate campaigns). ROAS reflects near-term financial consequences, placing at a medium construal level. Evaluation follows the same protocol as CTR: k-way accuracy, PE@ ,K, and error metrics (MAPE, MPE, RMSE).
High- Coggt-Cnntt Tasks, These tasks require abstract reasoning, long-term forecasting, inference of inter- nal states.
(1) Long-Term Video Memorability Prediction. Using the LAMBDA dataset (SI et al. 2023), persona agents predict how well video ads will be remembered over time, rescaled to a 0-10 range. Because this task involves forecasting retention beyond immediate perception, it reflects high-level construal. Evaluation uses Spearman's 's rank correlation (p), which measures agreement between the rank order of predicted recall scores and human annotations.
(2) Behavioral Attribute Classification (Reason, Persuasion, Emotion). These tasks involve deeper se- mantic reasoning: identifying an ad's justification (Reason), persuasive strategy (Persuasion), or evoked emotion (Emotion). We use the Image and Video Ads dataset (Hussain et al. 2017)) for reason and emotion, and the Video4096 dataset (Bhattacharyya et al. 2023) for persuasion. Evaluation uses accuracy, measured as exact match with ground truth.
Although traditionally applied prediction, we also test whether Social Agents can extend the wisdom of crowds to performative content generation by leveraging diverse social perspectives.
Tweet Content Generation. Social Agents generate tweets conditioned on an input advertisement. We use the test set of 2,339 questions from the LCBM dataset (Khandelwal et al. (0224), which includes 168 million multimodal tweets from 10,135 enterprise accounts (2007-2023) with associated like counts. Each persona agent independently generates a tweet with supporting rationale. A moderator agent, defined as an expert in tweet generation, synthesizes these into a consolidated tweet that reflects the collective social perspective, along with aggregated rationale. Evaluation uses BLEU-N (1-4) and ROUGE-1 I for lexical overlap with references, and G-Eval (Liu et al., 2023), a reference-free free evaluator that rates prompt alignment, factual reasonableness, and content suitability.
3 RESULTS AND DISCUSSION
3.1 EXPERIMENTAL SETUP
To test the model-agnostic nature of Social Agents, we evaluate it on three state-of-the-art LLMs: GPT-4o (Hurst et 2024), LLaMA 3.3 70B (Dubey et 2024), and Qwen3 32B (Yang et al. (202)). For Webpage Likability Prediction, which requires image understanding, we use their dision-langure
counterparts: GPT-
4o, LLaMA 3.2 70B Vision (Meta Al 2024), and Qwen2.5 VL. 72B (Bai al., 2025). We also test smaller backbones: LLaMA 3.1 8B and Qwen3 8B, LLaMA 3.2 1 1B Vision and Qwen2.5 VL. 7B to evaluate robustness under reduced capacity.
8

---

Published as a conference paper at ICLR 2026
Following the wisdom of crowds principle, our main comparison is between Social Agents and a -o-Noorsoos variant, where a single LLM expert agent is prompted as domain specialist. Both are evaluated in a 5-shot setting, except Behavioral Attribute Classification, which uses zero-shot for comparability. For all tasks, we retrieve the top-5 nearest examples (excluding the target item) with OpenAl's text text-embeddiing- model [OpenAl 2024), using them as few-shot exemplars to ground responses. To ensure a fair comparison, we enforce strict and uniform 300-token maximum generation budget for both the No-Persona (5-shot) baseline and the Social Agents framework across all tasks and models, so performance gains cannot be at- tributed to longer outputs but instead to the structured diversity and aggregation mechanism of Social Agents. We further compare Social Agents with task-specific expert models: LCBM (Khandelwal et al.) 2024) for Ad CTR, tweet content generation, and engagement prediction; Henry (Sl et al. 2023) for memorability; behavior-LLaVA (Singh et al.] 2025) for behavioral attribution; and an XGBoost baseline for ROAS and webpage likability prediction. All experiments run with temperature 0.85 encourage output diversity.
3.2 DISCUSSION
Social Agents is Model Agnostic. A key strength of Social Agents is that its benefits are not tied to specific model family or scale. Across nine vision and language backbones, both proprietary and open- source, we observe consistent improvements over the No-Persona baseline, averaging 23.9% across all tasks and models (Appendix (A.2.l). Larger models such as GPT-4o and Qwen3 32B consistently outperform task-specific experts, with Qwen3 32B achieving the strongest CTR performance and the best ROAS gains, improving PE@ 30 by 145.5% in the real estate domain (Table 6). In ablation studies, even smaller models such as LLaMA 8B and Qwen 7B, despite lower absolute accuracy, still show clear improvements over their no-persona counterparts (Appendix I.. This demonstrates that Social Agents provides model-agnostic benefits, with diversity and aggregation yielding consistent gains across scales.
Performance across prediction and generation tasks. Social Agents delivers its strongest improvements on low- and medium-level construal tasks, where judgments are immediate and behavior-oriented. On web- page likability prediction, it improves over the No-Persona baseline by 164.2% (Table 5 In Ad CTR prediction, it reduces error by 32.8% (Table while tweet engagement accuracy rises by 23.7% (Table 4). For the medium-leve ROAS task, gains are also substantial, with a 39.8% MAPE reduction and a 145.5% improvement in PE@21 (Table 6). In generative settings, Social Agents improves tweet quality by 18.7% (ROUGE-1), and 7.7% on G-Eval (Table R). These results suggest that diversity and aggregation provide the greatest benefits for near-term behavioral predictions and creative generation, which, much like human judgments, are easier assess lower construal levels (e.g., whether something is likable).
Performance on high construal level tasks. On high-level construal tasks requiring abstract reasoning and long-term forecasting, Social Agents produces more moderate but consistent improvements over baselines. In long-term video memorability prediction, it improves over No-Persona by 24.2% (Table 77. Notably, performance on the high-construal memorability task remains remarkably stable across decoding temper- atures: This tight grouping indicates that temperature-driven variability has minimal impact on outcomes, and Social Agents consistently outperforms the No-Persona baseline across temperature values. For behav- ioral attribution, gains include 11.6% on reason, 55.3% on persuasion, and 34.3% on emotion (all labels) (Table 9. These results highlight that while improvements are smaller on distant, cognitively demanding tasks, much as they are for humans, the framework reliably and robustly outperforms single-agent baselines.
Comparison of Social Agents vs trained expert models. We benehmark Social Agents against task- specific expert models trained large behavioral datasets. On low- and medium- level tasks, it often surpasses these experts: in Ad CTR prediction it outperforms fine-tuned LCBMs by 33.2% in MAPE re- duction (Table 3) while in webpage likability it exceeds XGBoost by 139--644% (Table 5) On ROAS, it improves 30 by 128.6% again outperforming XGBoost (Table (6). In tweet generation, it also sur- passes fine-tuned LCBMs on BLEU-2 and ROUGE-1 (Table 8) Gains are strongest with larger backbones
9

---

Published as a conference paper ICLR 2026
such as GPT-4o, while smaller models provide more modest improvements. High CLT tasks are harder: in long-term memorability, Social Agents improves over No-Persona but does not match Henry, trained on ded- icated memorability corpora (Table ()) In behavioral attribution, however, narrows the gap, outperforming Behavior-LLaVA (Zero-shot) with up to 55.3% gains on persuasion and over 30% on action and reason (Table D. Strikingly, these results are achieved with only 5-shot prompting, while expert baselines rely on hundreds of thousands to millions of curated training examples. Overall, Social Agents readily surpasses experts on low- and medium- tasks, though matching highly specialized models on distant, cognitively demanding ones remains difficult.
Contribution of Personas Social Agents. Figure A.3.3 shows that gains come from persona diversity rather than increasing inference calls. In Ad CTR prediction (Figure [aa, the No-Persona baseline, which averages repeated calls to one prompt, quickly plateaus with higher error. Likewise, the "Wisdom of the Silicon Crowd" "Schoenegger al. (2024), aggregating multiple LLMs without persona conditioning, un- derperforms compared to single model with diverse personas. Social Agents achieves lowest MAPE with N 10 20 personas before diminishing returns. In webpage likability (Figure (bb). the pattern repeats: repeated calls or multi-model ensembles fall short, while just N = 10 personas yield the strongest human alignment. This highlights that conditioning on demographic and psychographic profiles provides greater collective intelligence than brute-force scaling.
Correlation Social Agents with Human Judgments. On Webpage Likability Prediction, Social Agents' persona-conditiones predictions show strong aligmment with human ratings, particularly among younger demographics. GPT-4o achieves correlations of 0.71 for 18-24 males and r 0.69 for 18-24 fe- males, while correlations drop to 0.22-0.25 for 55+ groups, refecting limited training coverage (Appendix A..322.. Female personas align more closely than male counterparts within the same age bracket. GPT- 4o consistently outperforms models with weaker alignment with human preferences translates into lower task performance. These results suggest that as LLMs better capture demographic variation, Social Agents' collective predictions will become even
accurate.
Effect of temperature across construal levels. Performance remains stable across decoding temperatures for both high- and low-construal tasks (Tables and On long term memorability, GPT-4o under Social Agents consistently achieves 0.41 Spearman correlation across temperatures, compared to 0.33 for the No- Persona baseline. Qwen3-32B similarly improves to 0.35 versus 0.33, with only minor fluctuation across settings. On CTR prediction, GPT-4o remains around 47.5 percent MAPE across all tested temperatures (47.71, 47.53, 47.60 47.57), substantially outperforming the 75.70 percent No-Persona baseline. Qwen3- 32B shows the same pattern, achieving 43.31 percent versus 52.90 percent for No-Persone Across both construal levels, temperature adjustments produce only small numerical variation and do not close the gap between Social Agents and the baseline. The improvements therefore stem from structured persona-level diversity rather than stochastic decoding effects. More details Section A.8
4 CONCLUSION
We present Social Agents, a framework that adapts the "Wisdom of Crowds" to LLMs by instantiating diverse persona agents whose independent judgments are aggregated into collective outcomes. By utilizing demographic and psychographic attributes, the system surfaces heterogeneous social perspectives rather than single uniform response. Across eleven behavioral tasks and nine models, Social Agents consistently improves performance, demonstrating that collective intelligence not only enhances LLM predictions but also shows promise scalable paradigm for improving decision-make mmaiing and generation tasks.
10

---

Published as conference paper ICLR 2026
REFERENCES
Aristotle. The Politics. Oxford World's Classics. Oxford University Press, Oxford, 2009. ISBN
9780199538737.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint arXiv:2502.13925, 2025.
Jakob Bernoulli. Ars Conjectandi. Thumisiorum Fratrum, Basel, 1713. Original formulation of the Law of Large Numbers.
Aanisha Bhattacharyya, Yaman K Singla, Balaji Krishnamurthy, Rajiv Ratn Shah, and Changyou Chen. A video is worth 4096 tokens: Verbalize videos to understand them zero shot. Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 9822-9839, 2023.
Pew Research Center. Social media fact sheet. https: www. www.wwwwwwwwww.ww..o.... org/ internet/ saatt-ssheetsociall-meee a/ 2024. Accessed: 2025-08-03.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp. 785-794, 2016.
DeepSeek. Deepseck-rl: A 6b moe-based reasoning model. Technical report, Open Source, 2024.
Dorottya Demszky, Dana Movshovitz-Attiee Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. Goemotions: A dataset of fine-graines emotions. In Proceedings of the 58th annual meeting of the Association for Computational Linguistics, pp. 4040-4054, 2020.
Abhimanyu Dubey, Abhinay Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama herd of models. arXiv e-prints, pp. arXiv 2407, 2024.
Francis Galton. Vox populi. Nature, 75:450 1907.
Aaron Gokaslan and Vanya Cohen. Openwebtext: An open source recreation of the gpt-2 training dataset. Sheston00770penWeTTee---C--ooppp 2019. URL htttp:://itthui http://tttub..co///cpttttsssstt openwebtext]
Michael Haas et al. Analysis of data from corporate prediction market. Management Science, 63(5):
1617-1634, 2016. Study of Best Buy's internal "Tag Trade" market.
Thomas Hagen and Thomas Espeseth. Image memorability prediction with vision transformers. arXiv preprint
January 2023. doi: 10.488550/axxx.2330.06666 URL http:::t/aarrttttt
htq/abs/2301. 08647
R. H. Hooker. Mean or median. Nature, 75:487-488, 1907. doi: 10.1038/075487a0.
URL https:
wentture.com/arttte com wtt.m//attillee0055551..11
Tiancheng Hu and Nigel Collier. Quantifying the persona effect in LLM simulations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 10289-10307, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 18653/v ac200//
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint 1rXiv:2410.21276, 2024.
11

---

Published as a conference paper ICLR 2026
Zacem Hussain, Mingda Zhang, Xiaozhong Zhang, Keren Ye, Christopher Thomas, Zuha Agha, Nathan Ong, and Adriana Kovashka. Automatic understanding of image and video advertisements. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pp. 1705 1715, 2017.
Ashmit Khandelwal, Aditya Agrawal, Aanisha Bhattacharyya, Yaman Kumar, Somesh Singh, Uttaran Bhat- tacharya, Ishita Dasgupta, Stefano Petrangeli, Rajiv Ratn Shah, Changyou Chen, and Balaji Krishna- murthy. Large content and behavior models to understand, simulate, and optimize content and be- havior. In The Twelfth International Conference on Learning Representations, 2024. URL https: //openreview. wwwpn/eeieww.eeeffoooo.oee wtree//fooommmm/RRRdp
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding, 2023.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop, pp. 74 74-81, 2004.
Siyang Liu, Trisha Maturi, Bowen Yi, Siqi Shen, and Rada Mihalcea. The generation gap: Exploring age bias in the value systems of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 19617-19634, Miami, Florida, USA, November 2024.
Association for Computational Linguistics. doi: 10. (0.18653/11/0224.emm mai...0ai..444.4.. URL https:
//aclanthology.oom wtt.///g///22222 caammlp-maile 1094/
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:230 2023.
Meta AI. Llama 3.2: Revolutionizing Edge Al and Vision with Open, Customizable Models https://
ai.meta comm bl og/ lama- 3-mma-3-2-- connect -otteectt-224---0 wone---wiioo--eeeemmoo.. -obbideeeeeooiiio www.ioodellieeee......ee ces/ September 2024. Meta blog post.
OpenAl. OpenAl GPT-3.5 Turbo API [large language model]. httpes//pllaffoooooeeess.c
doca/moode http://moddll/ppt-222/ // 2023. Accessed: 2025-07-24.
OpenAL New embedding models and api updates. https://openas http///eaaa.commnndd//ae.
eew-dembedi model s- -- annd-api-uddates. January 2024. Blog post announcing the text-embeddingg ng-3 family of models.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311-318. Association for Computational Linguistics, 2002.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. volume pp. 9, 2019.
Katharina Reinecke and Krzysztof Z Gajos. Quantifying visual preferences around the world. In Proceedings the SIGCHI conference on human factors in computing systems, pp. 11-20, 2014.
Shibani Santurkar Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 20901- 29971-30004. PMLR, 2023.
Philipp Schoenegger, Inés T. Park, Peter S. Pospisil, Pinar E. Tuncel, and Mihacla van der Schaar. Wisdom of the silicon crowd: Lim ensemble prediction capabilities rival human crowd accuracy. Science Advances, 10 adp 528), 2024. doi: 10.1 10.11c/iadv...dpp52522. URL https: www. se ence. once.org//oo//... 1126/ wwwadviaivv.e.... adp1522
12

---

Published as a conference paper ICLR 2026
Melissa Schwartzberg. Aristotle and the judgment of the many: Equality, not collective quality. The Journal of Politics, 78(3), 2016. doi: 10.1086/685000. Published online May 19, 2016.
Harini SI, Somesh Singh, Yaman K Singla, Aanisha Bhattacharyya, Veeky Baths, Changyou Chen, Ra- jiv Ratn Shah, and Balaji Krishnamurthy. Long-term ad memorability: Understanding & generating memorable ads. arXiv preprint arXiv:2309.00375. 2023.
Somesh Kumar Singh, Harini S I, Yaman Kumar Singla, Changyou Chen, Rajiv Ratn Shah, Veeky Baths, and Balaji Krishnamurthy. Teaching human behavior improves content understanding abilities of vlms. In International Conference Learning Representations (ICLR) 2025 Poster. ICLR, 2025. URL https:
/ //openreview. wtt.s/wwwee/.forrreoeoee fest//formmiddee de
James Surowiecki. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations. Doubleday, 2004.
A. Tess, J. Smith, and R. Lee. Generative ai meets open-ended survey responses. SAGE Open, 15(4):1-20, 2025. doi: 10. 0011177004912111513273.....
Yaacov Trope and Nira Liberman. Construal-level theory of psychological distance. Psychological Review, 117(2):440-46) 2010. doi: 10.1037/a0018963
Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, and Yu Qiao. Internvideo: General video foundation models via generative and discriminative learning, 2022.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388 2025.
A APPENDIX
A.1 EXPERIMENTAL DETAILS
A.1.I PROMPTING PERSONAS IN SOCIAL AGENTS
We use structured prompts that clicit nuanced, persona-driver judgments. Each prompt grounds the persona agent in specific weegrraphcccpchhrrappppee profile, provides context, and defines the evaluation task. We can define the complete prompt P, for an agent as a structured concatenation of four key components:
P = (S) + Persona (D;) + Task(C, T) @ Format(F)
where S is seste instruction, D, is the unique demographic profile for agent i, C the task context including few-shot examples, T is the specific task goal, JF is the required output format, and denotes string concatenation.
The general structure of prompts is encapsulated in the template shown below. We this struc- ture to adapt Social Agents to rest of the eleven behavioral tasks by simply modifying the task-specific components while keeping the core persona agent descriptions consistent.
13

---

Published as a conference paper ICLR 2026
Prompt Structure
Persona Description (D,): You are a Demographic Actributes). [A detailed, harrative description of the persona's values, media consumption habits, cultural touchstones, and what typically influences their judgments and preferences.]
Task Context Goal (C, T): You are given 5 examples [stimuli] and their corresponding [metric] scores (on 0-100 0-10 scale). You now shown [stimulus].
Your task is to judge how [target attribute] this [stimulus] is likely to for [your demographic group]. [Further clarification
what cognitive or smotional factors to consider for the judgment.]
Output Format (F): Return: Reason: [Provide concise explanation for your judgment, rooted in your persona's s perspective.] Answer: [A numerical score in the speci fied range, e.g., 0-100 0-10, based on the task.] t You must include this score.
For our experiments, we instantiate a panel of 10 distinct personas based on cross-section of age and gender. These personas, summarized in Table 2 are used across all evaluation tasks. The only exception is for the Long-Term Video Memorability Prediction task. To align with the demographics the original dataset's human annotators, female, 18-24 male, 25-34 female and 25-34 male.
more focused panel consisting of only the first four personas: 18-24
A.1.2 DETAILED TASKS AND METRIC FORMULATION
We provide the mathematical formulations the evaluation metrics used in our study. Throughout, an item refers to ad, webpage, video depending the task. Predictions produced by the system under evaluation (c.g., our Social Agents, or the No-Persona baseline). For Social Agents, persona outputs are aggregated by majority vote for categorical predictions in the Tweet Engagement Prediction and Behavioral Attribute Classification tasks, while for rest the tasks, the predictions are averaged using an aggregator before evaluation. The full mapping of tasks their construal levels with justification and evaluation metries is summarized in [
Accuracy: Accuracy measures the fraction of items (ads, webpages, or videos) for which the system's 's final human-provides provided label. It is defined as:
predicted label matches the human-
Accuracy Σ [w: De).
where y/ the human label (e.g., High or Low engagement for a tweet, High or Low likability score for a webpage, behavioral attribute for video) and ÿs is the system's prediction for item i.
For Webpage Likability Prediction, the averaged predicted likability score (0-10) is converted into a binary label using threshold of 5 (High if score 5, Low if < 5). This predicted category is then compared against the human-rated category. For Tweet Engagement Prediction, each persona agent outputs a binary label (High Low engagement), and the final system prediction is obtained by majority voting across personas. In case of ties in Social Agents, the framework is re-run for that tweet to resolve the decision. For Behavioral Attribute Classification on videos, the prediction is correct only if the chosen attribute label (one of Topic, Action, Reason, Persuasion, or Emotion) exactly matches one of the five human annotated label.
For discretized continuous prediction tasks on ads, such CTR ROAS percentile estimation, also report k-wayy accuracy. In this setting, both the ground-truth percentile and the predicted percentile
14

---

Published as conference paper ICLR 2026
Task Name
Construal Level
Construal Level. Justification
Metrics Used
Tweet Engagerennt Pre-
Low
Judging likes from surface tweet cues;
fast, low-
Accurucy
diction
abstraction decision.
Ad Click-Thrugh Rate
Low
Predicts immediate clicks based on perceptual and short- horizon content cues.
k-way Accuracy, PEAK,
(CTR) Prediction
MAPE, MPE, RMSE
Webpage
Likubility
Low
Judgments rely immediate perceptual features (e.g. color, layout, and visual hierurchy); requires rapid, surface- leeeel evaluation.
Pearson Correlation. Accu-
Prediction
racy, MAPE, MPE, RMSE
Behavioral Attrihute
Low
Topic can he identified directly from explicit video ad text visuals, without abstraction or long-term reasoning. Detects explicit calls- to-action (e.g. "Buy now") that visible on the surface of the video ad; minimal inference required.
Accuracy
Classification (Topic)
Behavioral Attribute
Low
Accuracy
Classification (Action)
Return
Spend
Medium
Involves forecasting near-term financial outcomes; re- quires reasoning about consumer response and cost ef- fectiveness beyond immediate perceptual cues. Requires predicting whether content will be recalled memory
k-way Accuracy. PE@ MAPE, MPE. RMSE
(ROAS) Prediction
Long-Term Video Memorability Predic-
High
Spearman Correlation
weeks demands abstract reasoning processes, semantic ussociations, and persuasive impuct. Goes beyond surface features to infor latent intent, per- suasive strategies, and emotional resonance, which re- quite deeper abstraction.
tion Behavioral Attribute High
Accuracy
Classification (Reason,
Persuasion, Emotion)
Tweet Content Genera- tion
BLEU-N (1-4), ROUGE-I,
G-Eval
Table I: Tasks, their construal levels. justificctti,,,,nnn,, and evaluation metrics. Low-level tasks focus on imme- diate, perceptual judgments; medium-level tasks address near-term outcomes; and high-level tasks involve abstract reasoning long-horizon forecasting. The tweet content generation task is not associated with con- strual levels, as involves producing performative content or text rather than making predictive judgments.
discretized into l equal-width bins (e.g., ht or = 10). A prediction is counted correct if the predicted bin matches the actual bin:
Accuracy koway kwway
Σ ((bi((A)) bin(P.)].
where A, is the actual percentile and P. is the predicted percentile for item i.
BLEU and ROUGE (Papineni al., 2002; Lin 2004) For the tweet content generation task, BLEU-N measures n- gram overlap precision between generated tweet and a human reference tweet, combined with a brevity penalty to avoid favoring short outputs:
BLEU = BP exp Σ wn log Pa
BP = exp(1 - r/c) c e
C
where Ph, the modified precision for n-grams and c, r are the predicted and reference lengths. We report BLEE-- through BLEU-4. ROUGE-1 reports unigram overlap between generated and reference tweets, with precision, recall, and F1 score. Standard normalization (lowercasing, punctuation stripping, tokenization) applied.
Mean Absolute Percentage Error (MAPE) and Mean Percentage Error (MPE): These metrics are used for numerical prediction tasks such as Ad Click-Through Rate (CTR) percentile, Return on Ad Spend (ROAS) percentile, and Webpage Likability Prediction. They capture the average proportional error magni- tude (MAPE) and the average signed bias (MPE):
MAPE
A
x 100%,
MPE
1
Σ E1
Ai
100%.
A
A.
155

---

Published as conference paper ICLR 2026
Here, A, is the human-provided ground-truth value for the 7th item (c.g., actual CTR percentile, actual ROAS percentile), and P; is the system's 's predicted value for the same item. MAPE reports the average absolute pereentage deviation, while MPE indicates whether the predictions systematically overestimate (positive) or underestimate (negative) relative to human ground truth.
Root Mean Squared Error (RMSE): Used webpage likability prediction task; RMSE summarizes the magnitude of prediction errors by penalizing larger deviations more strongly:
RMSE
Σ
(P. A.).
Here, n is the total number of items, A, is the human-provided ground-truth value for the 7th item, and Ps the system's 's predicted value for that item.
Pearson Correlation Coefficient (r: For webpage likability prediction task, Pearson's r measures the strength of linear association between system predictions and human provided ground truth across web- pages:
EL, A)(P: P) VAL - A)2 (P.
Ā
A P=
Σ P.
Here, A; is the human-provided ground-truth value for the 5th item, and P; is the corresponding system- predicted value for that same item. is the mean of all human-prn suman-provided ground-truth values across the webpages, and P is the mean of all system-predicted values across the webpages.
Spearman Rank Correlation (p): For long-ter term-termmmm video memorability prediction, Spearman's 's p measures whether the ranking of system- predice-redicted recall scores aligns with the ranking of human- measured recall scores for the videos:
6EP T1n122 1)
=
d, = (a,)(AA) rank(P:).
Here, A; is the human recall score, P; is the predicted recall score by the system, and d; is the difference between the ranks of these two scores for the 7th video.
Prediction Error at K (PE@ K): For ad CTR and ROAS percentile prediction, PB@ measures the per- centage of ads whose absolute percentage error falls within tolerance of K% of the ground-truth percentile:
PEOK 3
A
100
A
with E {10, 20, 30). Here, A, is the human human-provided ground-truth percentile for the 2h ad, P. the predicted percentile for that ad, and the indicator function counts an ad correct if its prediction error is within the specified tolerance K.
For the tweet content generation task, G-Eval uses large language model (GPT-4o) as an evaluator. It takes three inputs: the original ad prompt, the system-generated tweet, and one human reference tweet. Guided by a rubric, the evaluator assigns scores along three dimensions: (i) Prompt whether the generated tweet effectively addresses the intent or theme of the target context (i.e., the ad's intent, theme, or message), even if phrased differently from the human reference tweet: (ii) Factual Reasonableness: whether the generated tweet is factually accurate or at least plausible given the target context (i.e., the ad's intent, theme, or message), avoiding clear inaccuracies; and (iii) Content Suitability: whether the generated tweet is coherent, well-structured, clear, and appropriate for the Twitter platform in terms tone and brevity.
G-Eval (Liu et al. 2023):
Alignment:
Each dimension is scored separately, and the final G-Eval score is the average of the three sub-scores:
G-Eval Score &o (Scorcprompt alignment ScorCfccuual reasonableness ScorC suitablliyy
16

---

Published as conference paper ICLR 2026
Demographic Profile
Key Psychographic Traits
18-24 female
Values bold aesthetics, authenticity, and viral social media trends. Attuned to humor, shock value, memes, and gaming culture.
18-24 male
25-34 female
Values aesthetics, emotional resonance, and lifestyle relevance. Values clarity, style, and visuals related to aspiration and technology. Appreciates authenticity, emotional intelligence, and meaningful content.
25-34 male
35-44 female
35-44 male
Responds to visuals that are smart, grounded, and purpose-driven. Drawn to emotional clarity, purpose, and community-oriented themes, Appreciates thoughtful, honest visuals grounded in wisdom and reality. Resonates with warmth, purpose, legacy, and deep emotional connection. Values sincerity, legacy, and visuals with clear, meaningful value.
45-54 female
45-54 male
55+ female
55+ male
Table 2: Demographic Profiles and Psychographic Traits of the 10 Social Agent Personas. Each persona combines a demographic group (age group paired with gender) with its corresponding key psychographic traits. The table highlights the types of aesthetics, values, and content themes that each group is most likely prefer when evaluating or responding an ad, webpage, video.
This setup allows the evaluation to capture not only word overlap but also the overall quality and contextual appropriateness of the generated tweets.
A.2
EXPERIMENTAL RESULTS
A.2.1
PERFORMANCE OF SOCIAL AGENTS
Tables
and 9 summarize results across eleven behavioral prediction tasks. GPT-4o, as the strongest model, consistently outperforms both baselines and trained experts, whereas all models show consistent gains. By contrast, LLaMA, with weaker alignment to human preferences, struggles to match the performance of specialized expert models.
Model
Accuracy (%)
GPT-3.5 OOpenAl 2023
61.52
LCBBM (Finctuned on Twitter Data) (Khandelwal et al. 2024
85.99
I.CBM (Finetuned on Twitter Youtuhe Data) Khandelwal et al. 2024)
84.53
No-Persona (GPT-4o)
70.27
Social Agents (GPT-4o)
86.90
No-Persona (Qwen3 32B)
54.20
Social Agents (Qwen3 32B)
63.85
No-Persona (LLaMA 3.3 70B)
$1.50
Social Agents (LLaMA 3.3 70B)
63.75
Improvement of Social Agents over No-Persona
21.75%
Table Tweet Engagement Prediction. Performance evaluated using Accuracy (%, higher is better). Social Agents outperform the No-Persona baseline across all models, with individual improvements of 23.68% (GPT-4o), 17.78% (Qwen3 32B), and 23.79% (LLaMA 3.3 70B). Our approach achieves perfor- mance closely comparable to that of fine-tuned LCBMs, with Social Agents (GPT-4o) surpassing the aver- age LCBM performance by 1.92% (averaged across both variants). Positive gains are shown green. Best models are denoted in green and runner-ups in blue
17

---

Published as conference paper ICLR 2026
Model & Dataset
MAPE L 3-way accuracy (%)
10-way accuracy (%)
Creative & Real Estate Industries
Human Baseline
111.03
33.33
10.00
Creative Industry
XGBuost (Chen Guestrin, 2016
48.82
22.62
LCBM (Zero-shot) hKaaddel wal et al. 2024
80.21
71.75
43,90
LCBM (Finetuned) Khandelwal et al. 2024
75.67
70.68
42.77
No-Persona (LLaMA 3.3 70B)
85.80
53.97
29.96
Social Agents (LLaMA 3.3 70B)
62.44
69.00
25.20
No-Persona (Qwen3 32B)
59.13
70.40
39.00
Social Agents (Qwen3 32B)
43.31
75.70
29.60
No-Persona (GPT-4o)
72.45
40.04
18.91
Social Agents (GPT-40)
47.60
77.19
45.79
Improvement of Social Agents over No-Persona
28.2%
29.6%
4.8%
Improvement of Social Agents over LCBB
33.2%
9.1%
5.1%
Real Estate Industry
XGBoost (Chen Guestrin, 2016
48.90
60.92
24.27
No-Persona (Qwen3 32B)
63.36
26.10
Social Agents (Qwen3 32B)
50.79
65.90
23.40
No-Persona (LLaMA 3.3 70B)
75.14
63.40
27.30
Social Agents (LLaMA 3.3 70B)
55.59
55.60
19.30
No-Persona (GPT-4o)
70.27
61.50
23.20
Social Agents (GPT-40)
45.62
69.40
29.60
Improvement of Social Agents No-Persona
27.2%
6.8%
1.8%
Table Ad Click-Through Rate (CTR) Prediction. Results on datasets we constructed from Creative and Real Estate industries listed in the Forbes Fortune 500, evaluated using MAPE (lower is better) and 3-way and 10-way accuracy (higher is better). Social Agents reduce prediction error compared to No-Persona baselines, with average MAPE reductions of 26.6% (LLaMA 3.3 70B), 23.3% (Qwen3 32B), and 32.8% (GPT-4o) across both industries. Compared to LCBM, Social Agents achieve further 33.2% lower MAPE in the creative industry. We report the average improvements across all metries: Social Agents compared to No-Persona (averaged over all models) and Social Agents compared to LCBM (averaged over zero-shot and finetuned models). Positive gains shown in green and declines in red. Best models are highlighted green and runner-ups blue
18

---

Published as conference paper ICLR 2026
Model
Pearson Correlation (r)
MPE (%
RMSE 1
Accuracy (%)
XGBoost
0.67
19.70
0.72
77.55
No-Persona (LLaMA 3.2 90B Vision)
0.35
36.55
1.98
57.73
Social Agents (LLaMA 3.2 90B Vision)
0.61
25.94
1.31
56.70
No-Persona (Qwen2.5 VL 72B)
0.38
21.25
1.15
70.10
Social Agents (Qwen2.5 VL 72B)
0.59
19.69
1.03
73.20
No-Persona (GPT-44)
0.28
59.31
2.86
61.22
Social Agents (GPT-4o)
0.74
14.95
0.84
80.61
Improvement of Social Agents over No-Persona
95.84%
36.53%
37.76%
11.40%
Table 5: Webpage Likability Prediction. Performance evaluated using Pearson correlation (r, higher is better), Mean Percentage Error (MPE, lower is better), Root Mean Squared Error (RMSE, lower better), and Accuracy (%, higher is better). Social Agents consistently outperform No-Persona baselines across all metrics. Social Agents outperform the No-Persona baseline across all models, with individual improve- ments of 74.29% (LLaMA 3.2 90B Vision), 55.26% (Qwen2.5 VI. 72B), and 164.29% (GPT-40). Overall improvement percentages represent the relative improvement of Social Agents over No-Persona (averaged across all models) for each metric. Positive gains shown in green. Best models are denoted in green
and runner-ups blue
Model & Dataset
MAPE +
PE@ 10 f
PE@20
PE@30
Creative & Real Estate Industries
Human Baseline
104.70
0.071
0.176
0.307
Creative Industry
XGBoost (Chen & Guestrin 2016
52.12
0.08
0.17
0.26
No-Persona (LLaMA 3.3 70B)
81.15
0.20
0.31
0.42
Social Agents (LLaMA 3.3 70B)
74.57
0.12
0.24
0.34
No-Persona (Qwen3 32B)
52.90
0.18
0.36
0.47
Social Agents (Qwen3 32B)
43.57
0.16
0.32
0.45
No-Persona (GPT-4o)
75.70
0.16
0,32
0.42
Social Agents (GPT-4o)
58.76
0.27
0.47
0.59
Improvement of Social Agents over No-Persona
17.7%
-7.4%
10.6%
5.1%
Real Estate Industry
XGBoost (Chen & Guestrin 2016
48.63
0.13
0.26
0.39
No-Persona (I.aMM 3.3 70B)
134.13
0.03
0.09
0.12
Social Agents (LLaMA 3.3 70B)
125.19
0.06
0.11
0.17
No-Persona (Qwen3 32B)
105.91
0.02
0.03
0.04
Social Agents (Qwen3 32B)
80.21
0.04
0.09
0.15
No-Persona (GPT-4o)
199.93
0.06
0.10
0.14
Social Agents (GPT-40)
50.23
0.15
0.32
0.46
Improvement of Social Agents over No-Persona
39.8%
136.4%
145.5%
128.6%
Table Return on Ad Spend (ROAS) Prediction. Results on datasets we constructed from Creative and Real Estate industries listed in the Forbes Fortune 500, evaluated using MAPE (lower is better) and PE@ K (higher better). Social Agents reduce prediction error compared to No-Persona baselines, with average MAPE reductions of 7.4% (LLaMA 3.3 70B), 20.95% (Qwen3 32B), and 48.65% (GPT-40) across both industries. We report the average improvements across metries: Social Agents compared to No-Persona (averaged over all models). Positive gains are shown in green and declines in red, Best models denoted in green and runner-ups in blue
19

---

Published as conference paper ICLR 2026
Model
Spearman
Correlation (p)
Human Consistency
0.55
10-shot GPT-3.5 (OpenAl 2023
0.06
Regression using VIT Teats MVTT (Hagen & Espeseth 2023
0.08
Henry trained on individual datasets (Singh et al. 2025)
0.55
Henry trained all (combined) datascts (Singh al. 2025
0.52
No-Persome (LLaMA 3.3 70B)
0.26
Social Agents (LLaMA 3.3 70B)
0.30
No-Persona (Qwen3 32B)
0.33
Social Agents (Qwen3 32B)
0.33
No-Persona (GPT-40)
0.33
Social Agents (GPT-4o)
0.41
Improvement of Social Agents over No-Persona
13.2%
Table Long-Term Video Memorability (LAMBDA) Prediction. Performance evaluated using Spear- man correlation (p, higher is better). Among non-trained methods, Social Agents (GPT-4o) shows the best performance, highlighted in yellow Social Agents outperform No-Persona baselines with individual im- provements of 15.38% (LLaMA 3.3 70B), 0.00% (Qwen3 32B), and 24.24% (GPT-4o). We report the average improvements: Social Agents compared to No-Persona (averaged over all models). Positive gains are shown in green and no improvement in gray. Best models are denoted in green and runner-ups in
blue
Model
BLEU-1
BLEU-21
BLEU-3
BLEU-11
ROUGE-11
G-Exal
(ICL. OpenAl
LCBM (Fineturte on TWitke Data) Khandeloal et al. 2024
66.15
50.50
37.71
25.95
15.91
LCBM (Finetuned Twitter + Youlbbe Daiy Khandowal et al. 2024
67.26
51.32
38.35
29.59
16.14
No-Persona (GPT-44)
32.90
23:77
14.57
Social Agents (GPT-lo)
70.31
54,54
40.38
31.71
14.89
No-Persom (Qwen3 328)
64.98
48.14
34.45
25.60
9.39
0.53
Social Agents (Qwen3 328)
66.43
49.73
36. 10
27.12
14.91
0.57
Social Agents (LJaMA 708)
55.87
40.70
28.64
21.01
Improvement of Social Agents over No-Persona
8.83%
10.12%
13.259
16.97%
18.72%
7.69%
Table Tweet Content Generation. Results across models measured via BLEU (1-4), ROUGE-1, and G-Eval metries (higher is better). Social Agents outperform No-Persona baselines across all models, with individual improvements of 33.40% (GPT-4o), 5.94% (Qwen3 32B), and 11.28% (LlaMA 3.3 70B) for BLEU-4; 2.20% (GPT-4o), 49.20% (Qwen3 32B), and 12.18% (LlaMA 3.3 70B) for ROUGE-I. Our ap- proach also outperforms the tuned LCBM baselines (averaged across both variants) on multiple metrics, with Social Agents (GPT-4o) achieving 7.14% improvement on BLEU-2 and 14.69% improvement on BLEU-4. Overall improvement percentages represent the relative improvement of Social Agents over No- Persona (averaged across all models) for each metric. Best performing models are highlighted green and runner-ups in blue
20

---

Published as conference paper ICLR 2026
Toplc
All labels
Clubbed
Random
2.63
3.37
14.3
8.37
3.34
3.34
Hussain al. (Pinetuned)
Huseain et al. 2017
35
32.8
48.45
Intem- Video (Finetuned)
$7.47
36.08
86.59
5.47
VideoChar (Zero-shoo) (Li
Video4499 (G5 Genernted Story GPT-3.5 Classifier)
Bbuttocharva et al. 2028
51.6
11.68
79.69
35.02
66.27
59.59
Vidbo4096 (GP Generated Story Flan and Classifier
Bhatochan barve er 2023
60.5
10.8
79.10
33.41
79.22
811772
Video4096 VViuna Generated Story Flas- Classifie))
Baurccharyya
2023
57.38
76.60
30.11
77.38
80.66
Story
Fincuurer [Shal baclar
Behavior LLaVA (Zero shot, w/ vidko + GPT-3.5 story) Singh el 2025
60.09
12.84
79,94
36.12
67.10
79.18
LLaVa (Finctuned, w/ video GPT-3.5 story)
Singh of 2023
71.2
39.55
86.17
65.03
80.44
81.67
No- Po-PPn (GPT-4o + GPT-lo Story) Improvement
70.76
16.21
61,53
50.20
91.20
80.75
Agents (GPT-40 GPT-40 Story)
92.30
Agents No-Persona
Improvement of Social Agents over behavior-LLLaV (Zero-hhot)
25.6%
34.3%
-22.7%
55.3%
37.6%
8.2%
No. Persona Qwenn 228 GPT-4o Story)
67.85
16.01
63.01
41.22
88.90
80.75
Social (Qwen3 32B GPT-4o Stary)
15.96
52.89
90.30
83.95
Improvement of Social Agents No-Persona
Improvement of Social Agents over behanior-LLLa) (Zer-shol)
15.6%
24.3%
-22.8%
46.4%
34.6%
6.0%
No-Persona (LIaMA 3.3 708 GPT-4o Story)
30.11
18.11
61.67
46.31
88.70
$1.80
Social Ageni (L.MAA 3.3 70B GPT-4o Story)
70.86
17.21
62.03
53.49
88.80
88.40
Improvement Social Agents No-Porsima
Improvement
Agents over Vehavtor-LLLaAA (Zero-shot)
Table Behavioral Attribution Classification. Performance evaluated using Accuracy (%, higher bet- ter) across five key dimensions: Topic, Emotion (All-labels and Clubbed), Persuasion, Action, and Reason. Social Agents show improvements upto 26.8% over No-Persona baselines, and outperform Behavior-LL (Zero-shot) by margins up 55.3%. However, we observe occasional declines Emotion elassification (both All-labels and Clubbed) for some configurations. Positive gains are highlighted in green, declines in
red. Best models are marked in green runner-ups in blue
21

---

Published as a conference paper ICLR 2026
A.3 ABLATION RESULTS
A.3.1 EVALUATION OF SOCIAL AGENTS ON LOWER PARAMETER MODELS
To assess whether compact, peerr-pareeeter models still capture the human preference signals required for Social Agents to function effectively, we evaluate such models on the Webpage Likability Prediction and Long-Tern Video Memorability Prediction tasks (Table T0
Model and Method
Spearman Correlation
Model and Method
Pearson Correlation
MPE
RMSE
Accuracy
LLaMA 3.1 8B
No- No-Persona
0.08
LLMMA 322 11B Vidio
No. Persona
0,02
No-Persona (Mean of 10 Trials)
0.12
No Persona (Mean of 10 Thals)
Secial Agents
0.38
0.37
41.48
27.47
1.81
1.21
40.21
60.82
Social Agents
0.12
Owea2.5 YL
Qwen3 8B
No Persona
0.54
47.88
2,12
41.24
Trials)
No-Persona
0.21
Social Agents
0.58
27.54
1.34
63.27
No-Persona (Mean of 10 Trials)
0.19
Social Agents
0.26
Table 10: Performance comparison on the Web- page Likability Prediction task using lower- parameter models. Metries reported include Pear- son Correlation (t), Mean Percentage Error (MPE) (L), Root Mean Squared Error (RMSE) (L), and Ac- curacy (1). Qwen2.5 VL 7B with Social Agents out- performs all other configurations across all metries, achieving the highest correlation and accuracy, and the lowest MPE and RMSE. For LLaMA 3.2 1IB Vision, we find that the No-Persona (Mean of 10 Trials) base- line performs better than Social Agents, highlighting the limitation of these smaller models in understand- ing personas. Best- best-performing models are highlighted green and runner-ups in blue
Table 11: Performance comparison on the Long- Term Video Memorability task using lower- parameter models. Metrics reported include Spearman correlation (t). Qwen3 8B with So- cial Agents achieves the highest rank-order corre- lation with human judgments. For LLaMA 3.1 8B, observe that the No-Persona (Mean of 10 Tri- als) configuration performs comparably to the So- cial Agents variant, highlighting the model's lim- ited ability to interpret personas. Best-performing strategies are highlighted in green and runner-
ups in blue
the Webpage Likability Prediction task. Qwen2.5 VL 7B with Social Agents achieves 7% relative gain in correlation, a 15% reduction in mean percentage eтrоr (MPE), and 3% uplift in accuracy. LLaMA 3.2 11B Vision shows the largest error reduction, with 21% drop in MPE and 42% reduction in RMSE when using Social Agents compared to the No-Persona configuration. However, its best absolute performance correlation and accuracy is obtained by the No-Persona (Mean of 10 Trials) baseline rather than Social Agents, suggesting that some smaller models benefit primarily in error reduction while missing finer gains discrimination and alignment with human judgments.
In the Long-Term Video Memorability Prediction task, Qwen3 8B with Social Agents improves perfor- mance by 24% over the No-Persona baseline, while LLaMA 3.1 8B yields similar moderate gains across both No-Persona (Mean of 10 Trials) and Social Agents configurations.
Overall, Social Agents generally outperform the No-Persona bascline, though the magnitude of improvement varies. Gains tend to be smaller inconsistent in lower-capacity models, reflecting limits in model expres- sivity and reduced exposure to diverse human data. Even so, Social Agents help narrow the performance gap to larger models by reducing prediction errors.
22

---

Published as conference paper ICLR 2026
A.3.2 ALIGNMENT OF SOCIAL AGENTS' PREDICTIONS WITH HUMAN JUDGMENTS
To measure how well simulated persona agents in the Social Agents framework reproduce real human pref- erences, the Webpage Likability Prediction task (Figure The dataset contains 98 webpages, each rated on a 10-point Likert scale by nearly 40,000 participants from 179 countries, providing diverse demo- graphic coverage. For evaluation, compute the Pearson correlation (r) between the likability scores pre- dicted by Social Agents persona demographic groups and the average human ratings from the corresponding demographic groups for each wehpage. Results reported across three different model backbones.
07
Pearson correlation between Hai and 5/n or N//o
0.40
0.53
0.69
Demographic Group d (Age Cender)
18-24 male
0.34
0.23
0.54
0.36
0.71
0.27
-08
25-34 fomale
0.40
0.33
0.55
0.40
0.67
0.33
Nolations
-95
Webpage
25-34
0.35
0.27
0.53
0:35
0.67
0.28
di specific demographic group (one age gender)
range paired
0.30
0.27
0.40
0.36
0.52
0.31
-04
Ha, i Mean of likability scores
wabpage givan by all human raters in demographic group
35-14 make
0.31
0.14
0.40
0.31
0.55
0.27
Sio Model m's persone-conditoned
46-54
0.29
0.06
0.30
0.35
0.41
0.26
likabiitty score prediction for demographic group webpage!
02
0.21
0.30
0.40
Nfm!) Model m' a no-persone likability
85+ female
0.16
0.12
0.22
0.36
0.32
0.14
score prediction webpage
85- male
0.16
0.22
0.22
0.27
0.25
0.20
000
LiaMA 3.2 908 Vision
Cwen2.5 VL 728
GPT-4o
Model with Social Agents and No-Persona (represented in Red border)
Figure 5: Correlation between Social Agents' predictions and human judgments across demographic groups on the Webpage Likability Prediction task. Stronger correlations observed in younger demo- graphic groups, while performance gradually declines in older groups, likely reflecting limited training data used from older populations for the models. Pearson correlation coefficients (r)) reported between human demographic group ratings and Social Agents' simulated demographic group predictions, shown for three models: LLaMA 3.2 90B, Qwen2.5 VL 72B, and GPT-4o.
GPT-4o most closely matches human preferences, achieving r = 0.71 for 18-24 male (i.e., 71% of the maximum attainable linear agreement between Social Agents' predictions and the average ratings from this demographic group) and r 0.69 for 18-24 female (69% agreement). Correlations decline steadily with age, with 55+ personas yielding r between 0.22-0.25 (22-25% agreement with human ratings from the cor- responding demographic groups). Across all models, female personas show consistently higher alignment, averaging 5-10% stronger correlations than male personas within the same age bracket. Similar trends are observed across the other models.
As noted in prior work on demographic performance disparities in LLMs (Liu et al. 2024), a likely expla- nation for the weaker correlations among older demographics is the training data composition. Most Large vision and language models are trained predominantly on large-scale web and social media corpora (sources heavily skewed toward younger, digitally native users). Supporting this, Pew Research Center data (Center 2024) show that only 19% of U.S. adults aged 65+ report using Instagram and just 10% use TikTok (com- pared with 76% and 59% of adults aged 18-29, respectively). This skew means the latent representations learned by models reflect the aesthetic norms and preferences of younger cohorts. Older users, whose vi- sual tastes are shaped by different cultural influences, design eras, and usage contexts, underrepresented
23

---

Published as conference paper ICLR 2026
in training, reducing the model's ability simulate their preferences and leading lower alignment with human judgments in those age groups.
A.3.3 IMPACT OF NUMBER OF PERSONAS AND CALLS ON PREDICTIVE PERFORMANCE
We study how predictive performance varies with the computational budget N, defined as the total number of model calls allocated per item (i.e., per webpage or per ad) in Figure 6a and 6b Budget be spent in two ways: by repeatedly calling the same model prompt (increasing the number of calls), or by expanding the diversity of simulated persona agents. This done by increasing the number of unique personas across demographic dimensions. We generate person agents by sampling from a broad set of demographic and psychographic dimensions designed to capture a wide spectrum of human experiences and viewpoints. These dimensions include:
Age: [18-29, 30-49, 50-64, 65+]
Education Level: [Less than high school, High school graduate, Some college (no degree), Asso- ciate's degree, College graduate/some postgraduate, Postgraduate]
Gender: [Male, Female]
Race/Ethnicity: [White, Black, Asian, Hispanic]
Annual Income: [< $30,000, S30K-S100K, > $100K]
Political Ideology: [Liberal, Moderate, Conservative]
Political Affiliation: [Democrat, Republican]
Religion: [Protestant, Jewish, Atheist, Muslim, Hindu]
Let D; be the number categories in demographic dimension The total potential persona space is:
8 PI = II D; = 4x6x2x4x
5 = 17,280.
From this universe 17,280 possible combinations, we curate and expand a subset of up to personas for our experiments.
To study how different allocations of this computational budget affect predictive performance, we compare the following five strategies:
No-Persona (Mean over N Calls): A No-Persona baseline with GPT-4o, where budget N is spent on N repeated calls using the same input prompt per webpage or ad.
Social Agents (10 Personas, N/10 Calls per Persona): A fixed, panel of 10 persona agents, where given budget N, the calls per webpage or ad are evenly distributed across the panel such that each persona queried N/10 times. For example, if N 20, each persona is queried twice (10x2 2 20 total calls).
Social Agents (N Personas): Here, number of persona agents and it's diversity scales directly with budget. For N 100, we instantiate N unique personas and query each once. Starting from base pool Phose of 10 personas (five age groups x two genders), we apply an expansion function,
Poese A A
where A is set of auxiliary demographic and trait attributes. Iteratively enriching Pruse yields up to 100 personas. For given budget N, we sample N distinct personas (e.g., if = 20, 20 unique persona prompts are generated and each is used once).
24

---

Published as a conference paper ICLR 2026
Pearson Correlation (r)
Locel Agant the ot Agent Heen over Peoone Aperti Cato Neoore Agert
Agm sumb-ite cinoso Pluu ovr of Nnue
Agents ofhenoe
Wideo KLUMLY1
Madomol the
Soc Dowd XCUMS 19 Posore Apents
Wikeod leau sultrree (cu.m.
Number inference Calls Persona Agents Webpage (N)
Number Inference Calls Persona Agents Weboage (N)
(a) Ad Click-Through Rate Prediction
(b) Webpage Likability Prediction
Figure 6: Impact of number of inference calls persona agents (N) two tasks. The r-axis denotes the per-item budget, realized either as independent calls to single prompt or as N persona agents with one call each for given ad webpage. (a) Ad Click-Through Rate Prediction. Evaluated using Mean Absolute Percentage Error (MAPE; lower is better). Performance platcaus by N x 20-30, indicating that increasing the budget beyond this yields negligible gains. The best MAPE is achieved by Social Agents with
N = 20 persona agents (also matched at N 100), using one call per persona. (b) Webpage Likability Prediction. Evaluated using Pearson correlation (r; higher is better). Performance plateaus around N {20, 30, 40), and the highest correlation is obtained by Social Agents with N = 10 persona agents and one call per persona.
Wisdom of the Silicon Crowd (WoSC) (Schoenegger al. 2024): K LLMS (N/K) Calls per LLM. An ensemble of K = 3 distinct LLMs, . = [GPT-4o, Qwen3 32B, LLaMA 3.3 70B): Each model receives [N/K] calls. For example, with N 20, each model is queried 6 times, yielding 2 N total calls. This setup tests the value of model diversity while keeping the total budget fixed.
WoSC Social Crowd: K LLMs x (N/K) Personas. A hybrid approach that combines model diversity with persona agent diversity. With K = 3 models and total budget N, each model is paired with N/K distinet persona agents, queried once each. This ensures exactly N persona agent-model calls per ad webpage.
Ad CTR Prediction Performance
As shown in Figure (a), the clearest gains emerge when scaling persona diversity. Social Agents (N Personas) variant consistently delivers the lowest mean absolute percentage error (MAPE), dropping from about 32% at N 10 to below 29% N 100. This steady decline highlights the value of incorporating an increasingly diverse set of personas, which broadens the range of simulated viewpoints and reduces systematic error. By contrast, the WoSC ensemble achieves a substantial reduction relative to the No-Persona baseline (mid 50% versus 771% MAPE), but exhibits little to further improvement demonstrating that model diversity alone plateaus quickly. The hybrid WoSC x Social Crowd settles near 39-40% MAPE, showing minimal benefit from larger budgets, splitting calls across multiple models dilutes the effective persona coverage and constrains overall gains.
N increases,
Webpage Likability Prediction Performance
Figure (b) show that Social Agents with 10 Personas achieve the strongest Pearson correlations of r se 0.74-0.76 across all budgets. This demonstrates that carefully curated, representative panel is especially effective for subjective, aesthetic judgments. The Social Agents (N Personas) variant also improves over both the No-Persona baseline (r z 0.56) and the WoSC ensemble (r z 0.35-0.55), but does not surpass the
25

---

Published as conference paper ICLR 2026
fixed 10 persona panel, suggesting that indiscriminate persona agents expansion may introduce noise rather than strengthen signal. The WoSC ensemble underperforms sharply, with unstable correlations that reflect the misalignment of some constituent models with this subjective task. The hybrid WoSC x Social Crowd provides modest improvements (r = 0.56-0.59) over WoSC alone, yet remains below both Social Agents configurations.
A.3.4 DIVERSITY ACROSS PEERS-P PREDICTIONS SOCIAL AGENTS
To understand how simulated personas differ in their individual perspectives, diction divergence on the Webpage Likability Prediction task using Wasserstein distance. Figure 7 presents heatmap of Wasserstein distances between predicted likability distributions for persona archetypes de- fined by age group combine with a gender. Fach cell quantifies the distributional difference between two demographic groups' predictions, with higher values indicating greater divergence.
inter- enter-persona pre-
18-24 female
0.00
0.08
0.26
0.39
0.35
0.78
0.68
0.83
0.56
0.78
Divergence between demograp group d
Demographic group d (Age Gender)
18-24 male
0.09
0.00
0.20
0.34
0.30
0.75
0.55
0.79
0.52
0.74
0.71
webpage likabiiity score predictions
25-34 female
0.28
0.28
0.00
0.37
0.25
0.85
0.48
0.70
0.40
0.64
0.59
25-34 male
0.89
0.23
35-44 female
0.47
35-44 male
0.79
0.45
0.00
0.22
0.10
0.36
45-54 female
0.68
0.98
0:48
0.80
0.31
0.22
0.00
0.25
0.12
0.20
0.24
45-54 male
0.03
0.79
0.70
0.51
0.53
0.11
0.26
0.00
0.34
0.13
85+ female
0.65
0.40
0.29
0.25
0.28
0.12
0.34
0.00
0.8
0.12
55+ male
0.78
0.46
-..00
18-24 female
18-24 male
25-34 female
25-34 male
35-44 female
35-44 male
45-54 female
45-54 male
65+ female
55+ male
Demographic group (column)
Figure Inter-Persona Prediction Divergence on the Webpage Likability Prediction task. The heatmap shows clear clustering by age, with adjacent age groups exhibiting lower divergence (lighter cells along the diagonal) and sharp separations between younger females and older male groups (darkest cells, up to 0.83). Divergences between male and female personas within the same age bracket are comparatively smaller. Each cell reports the pairwise Wasserstein distance between predicted webpage-lik likability prediction distributions for GPT-4o when prompted with personas from different age and gender groups, where larger values indicate greater divergence in predictions.
We observe the following patterns:
Age-Based Divergence. Age is the dominant source of divergence. Younger personas (18-24) show only moderate differences with adjacent groups (divergences of 0.26-0.39 against 25-34), but diverge strongly from older cohorts. For instance, the 18-24 female persona reaches distances of 0.58-0.83 when compared with 45+ groups, reflecting clear generational gaps in aesthetic preferences.
26

---

Published as a conference paper ICLR 2026
Gender Effects. Within the same age group, male-femalc divergences are present but smaller, typically ranging from 0.09 to 0.34. This indicates that gender contributes to variation in predictions.
Extreme Divergence Cases. The most pronounced divergence (0.83) occurs between 18-24 females and 45- 54 males, reflecting maximal separation across both age and gender. The 45-54 male persona also shows con- sistently high divergence from younger groups (0. (0.70-0.83). By contrast, older males (55+) display slightly lower but still substantial divergence (0.64-0.78) from younger females, remaining significantly higher than within age comparisons.
A.4 EFFECT OF AGGREGATION METHOD IN SOCIAL AGENTS
James Surowiecki's account of the Wisdom of Crowds identifies four core ingredients: diversity, indepen- dence, decentralization, and a mechanism for aggregation. While the literature often defaults to median aggregation for robustness, Hooker (Hooker, 1907) reanalyzed Galton's 's ox-weight experiment and demon- strated that the mean can be equally reliable, and in structured settings even superior. Mean aggregation is not inherently weaker; effectiveness depends on the structure of the underlying sample.
Our Social Agents framework more closely resembles this structured case. Personas are not arbitrary guesses but controlled social perspectives with systematic variation. Because the prediction set organized rather than noisy, we treat aggregation as tunable hyperparameter and directly compare mean and median across tasks. In every evaluation, the mean performs better, although both aggregation methods substantially out- perform the No-Persona bascline.
On ad CTR prediction, mean aggregation reduces GPT-4o error from 72.45 percent MAPE to 47.60 percent, while the median achieves 54.96 percent. On long-term video memorability prediction, the mean reaches 0.41 Spearman correlation compared to 0.37 for the median and 0.33 for No-Persona. A similar pattern appears in webpage likability prediction, where the mean attains 0.74 Pearson correlation and the median 0.71, both well above the 0.28 No-Persona baseline.
Ad CTR Prediction (MAPE L)
Model
MAPE (%)
XGBoost (baseline)
55.40
LCBM (Zero-shot)
80.21
LCBB (Finetuned)
75.67
No-Persona (GPT-4o)
72.45
Social Agents (Mean)
47.60
Social Agents (Median)
54.96
Table 12: Mean vs. median aggregation on Ad CTR prediction.
Long-Term Video Memorability (Spearman f
Model
Spcarman
Human Consistency (baselinc)
0.55
Henry (baseline)
0.52
No-Persona (GPT-4o)
0.33
Social Agents (Mean)
0.41
Social Agents (Median)
0.37
Table 13: Mean median aggregation on long term memorability prediction.
27

---

Published as a conference paper ICLR 2026
Webpage Likability (Pearson ?)
Model
Pearson
XGBoost (baseline)
0.67
No-Persona (GPT-4o)
0.28
Social Agents (Mean)
0.74
Social Agents (Median)
0.71
Table 14: Mean vs. median aggregation on webpage likability prediction.
Overall, when diversity is structured rather than random, mean aggregation more effectively captures dis- tributed signal. At the same time, both mean and median aggregation consistently outperform single-agent baselines, underscoring the robustness of the Social Agents framework.
A.5 EFFECT OF PERSONA DISTRIBUTION AND DEMOGRAPHIC GROUNDING IN SOCIAL. AGENTS
We design personas by grounding each agent in demographic and psychographic dimensions commonly used in the behavioral and social sciences, including age, gender, education level, race and ethnicity, political ideology, political affiliation, income level, and religious background. In our main experiments, we use ten personas varied primarily by age and gender. We also conduct extended ablations in which personas are sampled across a broader multidimensional space.
The full space spans eight factors: age (18-29,30 30-49, 50-64, 65+), education (less than high school through postgraduate), gender (male, female), race and ethnicity (White, Black, Asian, Hispanic), income (less than 30K, 30K-100K, more than 100K), political idcology (liberal, moderate, conservative), political affiliation (Democrat, Republican), and religion (Protestant, Jewish, atheist, Muslim, Hindu). This yields a theoretical universe of 17,280 persona combinations. From this universe, we curate subsets for experimentation. Scaling analyses show that performance improves as the number of agents increases from three to ten to twenty, but stabilizes beyond that point. Once major clusters of social viewpoints covered, additional personas tend to introduce overlapping perspectives rather than new signal, and the marginal gains plateau.
Whether personas should match real-world demographic proportions depends the task and data avail- ability. Most benchmark datasets provide only aggregate labels without demographic annotations. In such low-resource settings, grounding empirical population proportions is feasible because there are no cittogrrphicccondiiinee labels to align with. The objective is therefore not to mirror population frequen- cies, but to ensure coverage of diverse viewpoints. This consistent with the Wisdom of Crowds principle, where diversity of opinion is the key requirement, while independence, decentralization, and aggregation are already enforced in our framework.
In high-resource settings where demographic labels are available, we explicitly test grounded persona condi- tioning. the webpage likability prediction task, we provide five-shot examples whose labels correspond to the specific demographic group represented by each persona. This yields modest but consistent improve- ment: the standard Social Agents configuration achieves Pearson correlation of 0.74 and 80.61 percent grounding increases performance to 0.7574 correlation and 86.31
accuracy, while demographie-condiioo percent accuracy. The improvement is incremental rather than dramatic, suggesting that broad structured diversity captures most of the benefit, with fine- fineinrineed demographic grounding offering additional gains when supported by labeled data.
Overall, findings indicate that structured persona diversity is sufficient activate the Wisdom of Crowds effect in low-resource conditions, while demographic grounding can further enhance performance when individual-leve demographic labels are available.
28

---

Published as a conference paper ICLR 2026
A.6 BIMODALITY IN BASELINE EVALUATIONS.
Human ratings in our datasets exhibit the expected unimodal, approximately normal distributions character- istic of aggregated judgments. In contrast, model outputs under the No-Persona baseline display a markedly different pattern.
For webpage likability prediction, the No-Persona baseline shows clear bimodality. We extend this analysis additional tasks and models under the same experimental setup. Using the mean of ten independent trials for GPT-4o, where the identical prompt is evaluated ten times and averaged, we observe bimodal prediction distributions across all three tasks: ad CTR prediction, long-term video memorability prediction, and webpage likability prediction. This suggests that unconditioned model does not sample from smooth continuum of viewpoints but instead oscillates between two dominant response modes.
Model
Ad CTR Prediction
Long-Tern Term Video Memorability
Webpage Likability
GPT-4o (No Persona)
Bimodal
Bimodal
Bimodal
Qwen (No Persona)
Unimodal
Bimodal or Multimodal
Bimodal
Table 15: Distributional structure of No-Persons baseline predictions across tasks.
For Qwen, long-term video memorability displays bimodal or multimodal structure, and webpage likability is also bimodal. The only exception is ad CTR prediction, where Qwen produces a unimodal distribution.
Overall, while human judgments form smooth unimodal distributions, No-Persona LLM outputs frequently cluster into two peaks across tasks and models, highlighting structural difference between human crowd judgments and neutral single-agent model sampling.
A.7 CROWDS WITHIN VS. SOCIAL AGENTS
The concept of "crowds within" suggests that repeated estimates from single individual approximate a Wisdom of Crowds effect. Prior behavioral research demonstrates that when an individual is queried multiple times under varied framings or contexts, internal variability can partially replicate the benefits of aggregation. Recent work on LLMs reports similar findings, showing that repeated prompting with different sampling conditions can yield modest performance gains.
To examine this mechanism, we implement a crowds-within condition by using two personas and inducing additional variability through different decoding temperatures. Although this approach improves over a single No-Persona call, the gains are substantially smaller than those achieved through structured persona diversity in Social Agents.
On long-term video memorability prediction, GPT-4o reaches a Spearman correlation of 0.20 under the crowds-within setup, compared to 0.41 with Social Agents using ten personas. Qwen3-32B attains 0.12 under crowds within, compared to 0.33 with Social Agents. A similar pattern appears in ad CTR prediction: GPT-4o achieves 56.65 percent MAPE under crowds within, compared to 47.60 percent with Social Agents; Qwen3-32B reaches 46.52 percent versus 43.31 percent under Social Agents.
These findings suggest that crowds-within approaches rely primarily on stochastic variation around a shared underlying viewpoint. In contrast, Social Agents introduces structured heterogeneity across distinct demo- graphic and psychographic perspectives. This form of diversity more closely aligns with classical Wisdom of Crowds conditions. As shown in scaling experiments, is introduced, performance gains plateau, indicating that the latent diversity of the model has been effectively covered.
sufficient number of distinct viewpoints
29

---

Published as conference paper ICLR 2026
Ad CTR Prediction
Long-Terna Video Memorability
Model
MАPE (%)
Model
Spearman
XGBoost
55.40
Human Consistency
0.55
LCBM (Zero-shot)
80.21
Henry (baseline)
0.52
LCBM (Finetuned)
75.67
Na-Persona (GPT-40)
0.33
No-Persona (GPT-40)
72,45
Social Agents (GPT-40)
0.41
Social Agents (GPT-4o)
47.60
No-Persone (Qwen3-32B)
0.33
No-Persona (Qwen3-32B)
59.13
Social Agents (Qwen3-32B)
0.33
Social Agents (Qwen3-33)
43.31
GPT-4o (Crowds Within)
0.20
GPT--4o (Crowds Within)
56.65
Qwen3-32B (Crowds Within)
0.12
Qwen3-32B (Crowds Within)
46.52
Table 16: Comparison Crowds Within and Social Agents across tasks.
A.8 ROBUSTNESS ACROSS TEMPERATURE SETTINGS
We evaluate Social Agents across range of decoding temperatures on one representative high-construal task, long-term video memorability prediction, and one representative low-construal task, ad CTR predic- tion.
On the high- ligh-conttrual memorability task, performance remains remarkably stable across temperatures. For GPT-40, Spearman correlations vary only slightly, from 0.407 at temperature 0.3, to 0.400 at 0.5, to 0.410 at 0.85, and 0.4112 12 0.95. Qwen3-32B exhibits similarly narrow range, between 0.3178 and 0.3489. Social Agents consistently outperform the No-Persona baseline at every temperature, and the results remain tightly clustered.
Long- Terg Video Memorability (Spearman p)
Temperature
GPT-4o
Qwen3-32B
0.40
0.33
0.5
0.40
0,32
0.85
0.41
0.33
0.9
0.41
0.35
Baseline
0.33
0.33
Table 17: Social Agents performance across temperatures on high-construal memorability prediction.
We observe the same pattern the low-construal CTR prediction task. GPT-4o remains clustered around 47.5 percent MAPE across temperatures, while Qwen3-32B varies within a narrow band. In all cases, Social Agents substantially outperform the No-Persona baselines.
Taken together, these findings demonstrate that Social Agents are robust to temperature-basee variability across both construal levels. Adjusting the sampling temperature produces only minor fluctuations and does not alter the underlying decision structure. The gains of Social Agents arise from structured persona-level diversity rather than stochastic variation introduced through decoding. While higher temperatures may allow slightly greater creativity and occasional marginal gains, overall behavior remains stable across models and tasks.
30

---

Published as conference paper ICLR 2026
Ad CTR Prediction (MAPE L)
Temperature
GPT-4o
Qwen3-32B
0.3
47.71
45.67
0.5
47.53
44.86
0.85
47,60
43.31
0.9
47.57
46.65
Baseline
75.70
52.90
Table 18: Social Agents performance across temperatures on low-construal CTR prediction.
31


Reviews:
[{"review_id": "E2tptEYXvH", "reviewer": "Reviewer_VRiC", "summary": "This paper investigates the strength and applicability of wisdom-of-crowds judgement in a variety of practical applications, ranging from tasks such as tweet engagement prediction to behavioral attribute classification (which ostensibly require more complex inference and theory of mind). The authors situate their evaluation in a cognitively inspired framework, classifying the difficulty of inference tasks based on their difficulty with respect to construal level theory (CLT). Detailed analysis of the kinds of tasks on which people and models are aligned–as well as the demographics that are aligned with models on this task—provide a comprehensive overview of the strengths and weaknesses of wisdom-of-crowds and persona-based inference in LLMs more generally. The authors are generally effective in comparing wisdom of crowds inference to informative human and domain-specific model benchmarks, and sufficiently justify the use of the method over domain-specific models.", "questions": "I am curious about the bimodal distribution for no-persona 4o prediction errors as seen in figure 4. Do the authors have any speculation as to why this model consistently over/underestimates human judgements? This is not a weakness of the paper per se–if anything it illustrates the strength of persona seeding in the effectiveness of wisdom of crowds judgements–but I’m curious how robust this bimodal effect is across models, and why this might be the case (it would be nice to see what this distribution looks like for the other models evaluated)", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I initially found myself asking several questions about the feasibility of this method (wisdom of crowds) over existing approaches, and the extent to which LLM judgments are supported by human behavioral data. These concerns were thoroughly answered by the evaluations comparing LLM results to both human judgements and existing domain-specific/trained methods. Moreover, the authors extensively justified the use of persona/demographic seeding over base model judgments (figure 4 provides a clear illustration of the benefits for demographic seeding in aligning with human judgements). Smaller parameter model performance in this task is another interesting finding which helps to support the feasibility of wisdom of crowds inference over existing methods, and the ablation analysis provides useful information on the limitations of small models in persona-based role-playing. Overall, the paper is well written and situated within the existing literature on wisdom-of-crowds judgment and an existing paradigm in cognitive science (CLT).", "weaknesses": "The authors claim that there are smaller improvements for cognitively demanding tasks (per CLT), “much as they are for humans”. However, I am not sure that this is supported by the results shown. Is there any kind of evidence to support the claim that for these tasks, higher level CLT evaluations are more difficult for people? What would the baseline be in this case?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iZPWhoL06w", "reviewer": "Reviewer_dg7w", "summary": "This paper introduces Social Agents, a framework that brings the Wisdom of Crowds to LLMs by creating diverse, human-like persona LLM agents whose independent opinions are combined into one collective judgment. Each LLM persona evaluates content such as ads or webpages, and their aggregated responses consistently outperform single-model predictions. Across multiple tasks, Social Agents improves accuracy both on simple judgments and complex reasoning. The study shows that diversity, not just scale, makes LLMs more accurate, interpretable, and human-like in their predictions.", "questions": "•\tFigure 4. Shouldn’t the y axis title simply be “probability density”? The current captions and notes are pretty lengthy.\n\n•\tIn WoC literature, the standard way to aggregate is to use median rather than mean [2]. Why do you decide to use mean rather than median? Would the result and conclusion change if you use median? \n\n•\tWhat is your view on “crowds within”, where repeated estimates from a single individual can also produce a WOC effect? This has been shown effective in both human [3] and LLM [4]. \n\n•\tHow do you decide the persona distribution in the demographic space? To match actual human behavior, shouldn’t you also ground persona to the actual human’s persona distribution?\n\n\nReferences\n\n[2] Francis Galton. Vox populi. Nature, 75:450–451, 1907.\n\n[3] Edward Vul and Harold Pashler (2008). Measuring the crowd within: Probabilistic representations within individuals. Psychological Science, 19(7):645–647.\n\n[4] Chuang, Y.-S., Narendran, S., Harlalka, N., Cheung, A., Gao, S., Suresh, S., Hu, J., & Rogers, T. T. (2025). Probing LLM world model: Enhancing guesstimation with wisdom of crowds decoding. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2025).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "•\tIn general, the paper provides great context on the importance and value of the study.\n\n•\tThe implications of the study are well articulated.\n\n•\tThe experiments are very comprehensive, spanning across 8 different domains and multiple datasets. \n\n•\tIncluding task-specific trained models is a wise move, which shows the usability of the proposed approach in the real world.\n\n•\tThe explanation of the theory and how that motivates the experiment design is well established.\n\n•\tThe figures 1 and 3 are well polished and convey the idea clearly.", "weaknesses": "•\tMissing literature. Previous studies have shown that LLM’s wisdom of crowds (WoC) do not always align with human’s WoC [1]. For some questions, their WoC shows human-like WoC but not for others. You should acknowledge this limitation of LLM-based WoC when making your claims.\n\n•\tThe baseline “5-shot best model” may be unfair. You should also control for the number of tokens in a generation. It is possible that the gain of social agents over the 5-shot baseline is merely due to the LLM using more tokens in their responses when given a persona.\n\n•\tGiven that the temperature value controls the variability in response, which is core to WoC, you conduct a sensitivity analysis on temperature, rather than fixating on a single temperature value.\n\n•\tThe structure of the paper is weird. Out of the 9 pages, the “result and discussion section” (section) only takes about one page,  and the subsection Section 3.1 “experimental setup” has nothing to do with results and I think should better belong to Section 2 “Setup”.\n\n•\tThere is a lack of statistical tests for all the claims made in the paper. Figure 3 should ideally also show standard errors/confidence intervals to account for noise.\n\nReferences\n\n[1] Chuang, Y. S., Harlalka, N., Suresh, S., Goyal, A., Hawkins, R., Yang, S., ... & Rogers, T. T. (2024). The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 46).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YjWd7IjXmJ", "reviewer": "Reviewer_JpnE", "summary": "The paper propose a special ensemble method called social agents, where the llm prediction is aggregated across different persona based predictions. The author find this greatly enhance the performance of various downstream opinion based tasks like twitter likability.", "questions": "1. Why can't the original LLM be a good enough predictor for collective opinion?\n\n2. Do you have explanation why only 10 personas boost the results by a huge amount of gap comparing to base llm? And could you try different temperature and see if the baseline has better prediction as well? \n\n3. See weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The idea of using persona conditioned ensemble of the same base llm is interesting and the experiments are comprehensive. The author also shows that the performance gains are agnostic of different models, which increase the applicability of the method.\n\n2. Social agent even beat trained expert which is interesting. And the performances on the benchmarks are pretty astonishing.", "weaknesses": "1. Motivation. Ideally LLMs that are trained on all website data is already representative of the collective opinion of the human population. The rlhf process might make the LLM a bit more biased and politically lean left, but in general it should already represent the general society. Most personalization paper wants to tailor to each individual opinions and stay away from collective opinions. The authors seems to be take a big circle and use personalization and aggregate to get the collective opinion, which is perplexing to me.\n\n2. Baseline choices and persona choices. For baseline, the author mentioned that they just sample from the same model N times. One would at least expect some sort of better ensemble here such as changing the temperature. For the social agent, the authors uses at most 10 personas. This is clearly not enough to represent the society. And the author mentioned diminishing return when increasing more persona, which is surprising.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)