Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View

## Abstract

Abstract Large language models (LLMs) have been shown to face hallucination issues due to the data they trained on often containing human bias; whether this is reflected in the decision-making process of LLM agents remains under-explored. As LLM Agents are increasingly employed in intricate social environments, a pressing and natural question emerges: Can LLM Agents leverage hallucinations to mirror human cognitive biases, thus exhibiting irrational social intelligence? In this paper, we probe the irrational behavior among contemporary LLM agents by melding practical social science experiments with theoretical insights. Specifically, We propose CogMir , an open-ended Multi-LLM Agents framework that utilizes hallucination properties to assess and enhance LLM Agents’ social intelligence through cognitive biases. Experimental results on CogMir subsets show that LLM Agents and humans exhibit high consistency in irrational and prosocial decision-making under uncertain conditions, underscoring the prosociality of LLM Agents as social entities, and highlighting the significance of hallucination properties. Additionally, CogMir framework demonstrates its potential as a valuable platform for encouraging more research into the social intelligence of LLM Agents.

## 1 Introduction

Human mind may often be better than rational.
– Leda Cosmides, John Tooby. With the extensive deployment of large language models (LLMs)[, ], LLM-based agent systems are increasingly developed to cater to diverse applications such as task-solving, evaluation, and simulation [, , , , ].
Given the similarities between the operational dynamics of LLM-based agent systems and human social structures, it is pertinent to explore the intersection of these domains.
Recent studies have highlighted the social potential of LLM Agents through constructing multi-agent systems that simulate interactive social scenarios [, , ]
revealing the social dynamics among interacting LLM Agents and showing parallels to human behaviors. For instance, LLMs can achieve social goals [] and adhere to social norms [] within LLM-based Multi-Agent systems.
Nonetheless, these research efforts exhibit two significant gaps:
1) They primarily focus on black-box testing in multi-agent role-playing systems, concentrating on the outputs and behaviors of agents while neglecting to investigate the internal mechanisms or cognitive processes that drive these behaviors.
2) LLM Agents are prone to hallucinations—producing misleading or incorrect information, due to their training data and inherent biases [, ]. The potential impact of such hallucinations on the social intelligence of LLM Agents remains under-explored.

Cognitive biases, pervasive in human society, highlight the subjective nature of human behavior [, ]. Human cognitive biases can lead to irrational decisions and imaginary contents like the hallucination phenomenon in LLMs [, ]. However, evolutionary psychology suggests that rationality is unnatural; rather, human irrationality is an adaptive selected trait for navigating complex social environments [, ]. Analogically, in this paper, we argue that LLMs’ hallucination (or imagination) attributes are the fundamental condition that confers social intelligence on LLM Agents.
We explore the similarities in social potential between human cognitive biases and LLM Agent hallucination attributes for the first time, particularly in irrational decision-making, to analogically deduce the underlying reasons for LLM Agents’ possession of social intelligence.

To study LLM Agents’ potential for irrational social intelligence, we present CogMir, an open-ended and dynamic multi-agent framework designed specifically for evaluating, exploring, and explaining social intelligence for LLM Agents via systematic assessments of cognitive biases.
Specifically, the hallucinatory attributes of LLMs are exploited (i.e., via treating the cognitive bias as a manageable and interpretable factor) in CogMir to probe their social intelligence, so as to provide enhanced interpretability for LLM agents.
In addition, our proposed CogMir framework integrates sociological methodologies to abstract typical social structures and employ various Multi-Human-Agent Interaction Combinations and Communication Modes to interlink System Objects.
This integrative setup is designed to systematically encompass and simulate various cognitive bias scenarios, as depicted in Fig. [1](#S0.F1).
On the evaluation front, CogMir combines sociological assessments, manual discrimination, LLM assessments, and traditional AI discrimination techniques to realize a multidimensional assessment system. By using flexible module configurations from standardized sets, CogMir simplifies social architectures, enabling diverse applications in experimental simulations and evaluations.

Designed as an open-ended framework for continuous interpretative study, we provide multiple CogMir subset samples as examples. Existing assessments of various cognitive effects demonstrate that LLM agents exhibit a high degree of consistency with humans in prosocial cognitive biases and counter-intuitive phenomena. However, LLM Agents demonstrate a higher sensitivity to factors like certainty and social status than humans, exhibiting more variability in their decision-making biases under conditions of certainty and uncertainty. In contrast, human decision-making tends to be more consistent across these conditions. In summary, this paper makes the following contributions:

- •
We are the first to breach the black-box theoretical bottleneck of the Multi-LLM Agents’ social intelligence, by utilizing LLM Agent hallucination properties to mirror human cognitive biases as explanatory and controllable variables to systematically assess and explain LLM Agent’s social intelligence through an evolutionary sociology lens.
- •
We propose CogMir, an extensible, modularized, and dynamic Multi-LLM Agents framework for assessing, exploiting, and interpreting social intelligence via cognitive bias, aligned with social science methodologies.
- •
We offer diverse CogMir subsets and use cases to steer future research. Our experimental findings highlight the alignment and distinctions between LLM Agents and humans in the decision-making process.
- •
CogMir indicates that LLM Agents have pro-social behavior in irrational decision-making, emphasizing the significant role of hallucination properties in their social intelligence.

## 2 Related Work

Our work is inspired by interdisciplinary areas such as social sciences and evolutionary psychology.

LLM Hallucination & Cognitive Bias. Hallucination in LLMs occurs when they generate content that is not factually accurate, often arising from the reliance on patterns learned from biased training data or the model’s limitations in understanding context and accessing current information
[, ]. Such hallucinations might be beneficial in creative fields, where these models can act as “collaborative creative partners.” They offer innovative and inspiring outputs that can lead to the discovery of novel ideas and connections []. Concurrently, cognitive biases and evolutionary psychology offer essential perspectives on decision-making processes and prosocial behaviors, which can be analogously applied to explain the social intelligence of LLM Agents[, ].
In this work, through mirroring human cognitive bias, we suggest that the hallucination property of LLM is the basis for prosocial behavior in LLM Agents, representing a potential form of advanced intelligence.

LLM Agent Social Intelligence Evaluation.
Several benchmarks traditionally utilized for evaluating the social intelligence of artificial agents, such as SocialIQA [] and ToMi [], are increasingly being surpassed in difficulty as language models advance. In response to this trend, recent efforts have synthesized existing benchmarks and introduced innovative evaluation datasets specifically tailored for assessing LLM Agents [, , , ]. Despite the wide range of social intelligence types [], there is no standard workflow for investigating LLM Agents’ social intelligence. CogMir has developed an open and accessible workflow aligned with consensus-based approaches in social science, facilitating systematic testing and advancement of social intelligence in language models.

Multi-Agents Social System. Dialogue systems facilitate AI interactions, with task-oriented models focusing on specific tasks and open-domain systems designed for general conversation, often enhancing engagement by incorporating personal details and creating deep understanding []. Simulations with LLMs demonstrate their abilities to produce human-like social interactions by applying these models to tasks like collaborative software development [, , , , , , ]. Despite these advancements, exploration of why these models exhibit social capabilities remain limited. Our work tries to bridge this theoretical gap by drawing on research methods from human social evolution studies, thereby enhancing the interpretability of Multi-LLM Agents social systems.

## 3 CogMir: Multi-LLM Agents Framework On Cognitive Bias

Figure: Figure 2: CogMir Framework. The framework is structured around four essential objects: humans, LLM Agents, data, and discriminators. These objects interact within the framework to facilitate Multi-Human-Agent (Multi-H-A) interactions and evaluations. CogMir features two communication modes and five Multi-H-A interaction combinations, enabling varied configurations to suit diverse social experimental needs. CogMir offers mirror cognitive bias samples (Fig. [1](#S0.F1)) and dynamic use cases open for expansion. The framework is depicted in a left-to-right sequence.
Refer to caption: /html/2405.14744/assets/x2.png

In this section, we provide a detailed and modular overview of CogMir, organized into four main elements: environmental setting, framework structure, cognitive biases subsets, and illustrative use cases. These components are visually depicted in a left-to-right sequence in Fig. [2](#S3.F2).

### 3.1 Environmental Settings

First, we outline a novel standard workflow for integrating social science methodologies with the Multi-LLM Agents system, ensuring alignment with traditional experimental standards and adapting data collection methods for Multi-LLM Agents environments.

CogMir environment settings are benchmarked against standard social science experiments through a structured three-step process: Literature Search, Manual Selection, and LLM Agent Summarization.
A literature search pinpoints key social science experiments, which are then manually selected for relevance and replicability. LLM Agents adapt these for integration into the Multi-LLM Agents system within the CogMir framework. In the Mirror Settings process, data collection methods such as surveys and interviews are transformed into Human-LLM Agent Q&A. Methods like case studies and naturalistic observations are adapted to Multi-Human-Agent interaction scenarios.

Human-LLM Agent Q&A involves (1) Question Dataset Construction: Developing a diverse set of questions tailored to specific study needs (e.g. multiple-choice, fill-in-the-blank, etc.) (2) Q&A Scenario Design: Pairing the Question Datasets with scenarios that simulate real-world environments (controlled settings like a room to dynamic public spaces like squares or transit stations). (3) Prompt Engineering: Crafting appropriate prompts for the LLM Agents based on the scenario and question dataset. (4) Analysis of LLM Agent Responses: Evaluating the responses from LLM Agents.

Multi-Human-Agent Interaction involves (1) Interaction Combination Configuration: Adapting human-only social science settings to interactive environments that include humans and LLM Agents (e.g., in group discussion experiments, some human participants are replaced with LLM Agents).
(2) Role Assignment: Specific roles and behaviors are assigned to humans and LLM Agents. This assignment is guided by prompt engineering to ensure each participant acts according to social science experiment guidelines.
(3) Communication Mode Selection: Based on the original social science setting select suitable communication modes for interaction.
(4) Data Collection and Analysis: Gathering and analyzing data from these interactions (e.g. dialogue, decision-making, etc.).

### 3.2 Framework Structure

After establishing realistic social science experiment environments, the next step is to select essential components to support the above two mirror methods: Human-LLM Agent Q&A and Multi-Human-Agent Interaction. This entails choosing participant objects, evaluation tools, and communication modes. The CogMir framework is organized into modules for Required Objects, Communication Modes, and Interaction Combinations to meet these needs.

Required Object Sets. Required Object encompasses all potential participants and evaluators involved in the system. Participants include humans and LLM Agents, which allows for dynamic setups where either or both can be involved in interactions depending on the experiment’s requirements. Evaluators include humans, LLM Agents, datasets, and discriminators. Datasets are utilized to store and construct prompts about the experimental setup (e.g. experimental scenarios, character information, etc.), task description, and Q&A question set. Discriminators are specialized tools utilized to evaluate the social intelligence of LLM Agents, encompassing three main types: State-of-the-art technical metrics such as SimCSE, SelfCheck, and FactScore [, , ] for objective, quantitative assessment; Human discriminators that delve into nuanced and subjective aspects like prosocial understanding; and LLM Agent discriminators, which involve the use of other LLM Agents to assess and challenge responses from a subject LLM Agent.

Communication Modes Sets. Communication modes dictate the nature of interactions within different setups. We model the participants (humans or LLM Agents) as channels based on information theory [] to define two essential communication modes:

- •
Broadcast (or Parallel, $C=C_{1}+C_{2}+\ldots+C_{n}$) which enables a single sender to transmit a message to multiple receivers simultaneously.
- •
Point-to-point (or Series, $C=\min[C_{1},C_{2},\ldots,C_{n}]$) establishes communication between two specific entities at a time ($C$ denotes channel capacity).

Multi-H-A Interaction Combinations Sets. This module provides various combinations of Multi-Human-LLM Agent interactions, tailored to different social science experimental needs, the most frequently used combinations in social science settings include:

- •
Single-H-Single-A: One human interacting with one LLM Agent, predominantly used for human-agent question-answering tasks (e.g. survey, interview, etc. ).
- •
Single-H-Multi-A: One human interacts with multiple LLM Agents, where humans can be set as controlled variables to test Multi-LLM Agents’s social cognitive behaviors.
- •
Multi-H-Single-A: multiple humans interact with a single LLM Agent, which is suitable for assessing the impact of group dynamics, such as consensus or conflict.
- •
Multi-A: multiple agents interacting without human participation.
- •
Multi-H-Multi-A: multiple humans and multiple LLM Agents interaction, integrating elements from the previous setups to mimic complicated experimental interactions.

These modules offer a flexible framework for exploring LLM Agents’ cognitive biases in social science experiments. Researchers can customize their setups by mixing different components to examine specific hypotheses. We outline cognitive bias subsets as guidelines in the next section.

### 3.3 Cognitive Bias Subsets

We offer a collection of seven distinct Cognitive Bias Effects subsets, tailored for the analysis of LLM Agents’ irrational decision-making processes:
a) Herd Effect []: refers to the tendency of people to follow the actions of a larger group, often disregarding their own beliefs. b) Authority Effect []: involves people being more likely to comply with advice or instructions from someone perceived as an authority figure. c) Ban Franklin Effect []: suggests that a person who does someone else a favor is more likely to do another favor for that person, due to cognitive dissonance. d) Rumor Chain Effect []: describes how information tends to change and distort as it passes from person to person, often leading to misinformation. e) Gambler’s Fallacy []: refers to the incorrect belief that past events can influence the likelihood of something happening in the future in random processes. f) Confirmation Bias []: refers to the tendency to favor, seek out, and remember information that confirms one’s preexisting beliefs. g) Halo Effect []: occurs when a positive impression in one area influences a person’s perception in other areas, leading to biased judgments.

The Cognitive Bias Subsets are discussed in detail in Section [4](#S4).

### 3.4 Sample Use Cases

Building on the above environmental settings and framework structure, we introduce two Evaluation Metrics as sample use cases to assess and analyze experimental outcomes for the seven identified classic Cognitive Bias Subsets in CogMir:

- •
Q&A Bias Rate ($Rate_{Bqa}$): Quantifies the LLM Agent’s tendency to exhibit cognitive biases under controlled, diverse cognitive bias Q&A survey with Single-H-Single-A.
- •
Multi-H-A Bias Rate ($Rate_{Bmha}$): Quantifies the LLM Agent’s tendency to exhibit cognitive biases under simulation scenarios with different types of Multi-H-A interaction.

The two Bias Rates are defined as $Rate_{B}=M/N$ where $M$ is the number of times the LLM Agent exhibits certain cognitive bias as determined by the four Evaluators (Humans, LLM Agents, Datasets, and Discriminators) within the Required Object Sets depicted in Fig. [2](#S3.F2). $N$ is the total number of inquiries, where $N=p\times q$, $p$ represents the number of repetitions, and $q$ is the number of distinct queries. The selection of Evaluators varies across different subsets of cognitive biases, affecting the Q&A Bias Rate and Multi-H-A Bias Rate calculation processes involved.

The above two metrics are designed based on replicability and generalizability criteria [], offering the potential for further extension. Potential future works and limitations are explained in Appendix.

## 4 Experiments & Discussion

In this section, we categorize the seven tested Cognitive Bias Subsets into two groups: those with Pro-social tendencies and those without. For detailed model comparisons, prompts, settings, and dataset explanations, see Appendix. An overview of the experimental setup follows:

Selected LLM Models. We select seven state-of-the-art models to serve as participants and evaluation subjects within our framework, specifically: gpt-4-0125-preview[], gpt-3.5-turbo[], open-mixtral-8x7b[], mistral-medium-2312[], claude-2.0[], claude-3.0-opus[], and gemini-1.0-pro[]. All LLM Agents have a fixed temperature parameter of 1 with no model fine-tuning.

Constructed Datasets. Leveraging social science literature [] and existing AI social intelligence test datasets [, , , ], we developed three evaluation datasets—two sets of Multiple-Choice Questions (MCQ): Known MCQ and Unknown MCQ, and one short content dataset: Inform. Additionally, we constructed three open-ended prompt datasets for Multi-H-A experimental initialization, requiring targeted data augmentation or curation to meet specific task needs: CogScene, CogAction, and CogIdentity. Known MCQ contains 100 questions with answers known to all tested models, queried 50 times each for consistent responses (e.g., “In which country is New York?”). Unknown MCQ includes 100 questions with unknown answers, focused on future or hypothetical scenarios (e.g., weather predictions for a specific day in 2027). Inform contains 100 short contents designed to investigate potential biases during information dissemination. CogScene features 100 scenarios involving actions, such as “attending a job interview at a catering company.” CogAction includes 100 distinct complete actions, exemplified by “borrowing a tissue”, which is a sub-dataset of CogScene. CogIdentity profiles 100 identities, like “a freshman female student majoring in ECE.”

Evaluation Metrics. Metrics are developed based on various experimental scenarios and evaluators, leading to specific Bias Rate metrics. For example, to test a cognitive bias within a particular scenario [S] of the CogSence dataset using the Known MCQ dataset [K] in a Single-H-Single-A Q&A format ($Rate_{Bqa}$, refers to Section [3.4](#S3.SS4)), with human evaluation [H], it is represented as $Rate_{Bqa}[K][S][H]$. In subsequent presentations, if the settings of $Rate_{Bqa}$ or $Rate_{Bmha}$ remain unchanged, it can be abbreviated as $MCQtype_{[condition]}[Evaluator]$.

### 4.1 Pro-Social Cognitive Bias Subsets

Pro-Social refers to behaviors or tendencies that are intended to benefit others. In the context of cognitive biases, Pro-Social biases lead individuals to act in ways that are perceived as helpful, cooperative, or altruistic towards others in a community or social setting [].

Harmony: Herd (Bandwagon) Effect.
We replicate the classic Solomon Asch experiment through CogMir to assess the Herd Effect on LLM Agents, mirroring the variables Asch identified[]:

- •
Uniformity: Conformity peaked with unanimous errors but fell with one correct answer.
- •
Variation in Incorrect Answers: Any dissent in the group reduced conformity.
- •
Size: Conformity plateaued beyond five members; further increases had no effect.

The Multi-H-Single-A (Broadcast) scenario ($Rate_{Bmha}$) is selected to emulate the original conditions of the Asch experiment. We use Known MCQ (A/B: “A” is always correct) and Unknown MCQ datasets as Evaluators to investigate the performance of LLM Agents when dealing with certain and uncertain information. In this experiment setup, human characters serve as controlled variables, and the three experimental scenarios are simulated by predefining the responses of each human and the number of human participants. For each MCQ dataset, we query every MCQ 10 times, resulting in $10\times 100$ inquiries. As defined in Section [3.4](#S3.SS4), the experimental results of the Multi-H-A Bias Rate for the seven different model-based LLM Agents under various conditions are presented in Tab. [1](#S4.T1).

Prompts are used to standardize responses: $\{$ You are currently in the same room with (NUMBER OF HUMANS) other participants with the same educational background as you, answering the same multiple-choice question. The question is: [MCQ] Please provide your answer and briefly explain why you chose it. Reply in the following format: Answer: A/B. Explanation: ***.$\}$ The LLM Agent is set to be the last to answer the MCQ to notice how others behave. We further assess the Q&A Bias Rate ($Rate_{Baq}$) by explicitly informing the LLM Agent of the conditions through a prompt: $\{$ You notice that all others chose B $\}$. The statistical results align with the Multi-H-A Bias Rate.

**Table 1: Herd Effect $Rate_{Bmah}\%$ via Multi-H-Single-A (Broadcast). $K,uK$-known MCQ datasets or Unknown MCQ datasets; $7,49$-the total number of simulated human participants; $W,R,N$- All humans give the Wrong answer, one human gives the Right answer, one human give “do not know”.**
| Model | $K_{[7W]}$ | $K_{[7R]}$ | $K_{[7N]}$ | $K_{[49W]}$ | $uK_{[7W]}$ | $uK_{[7R]}$ | $uK_{[7N]}$ | $uK_{[49W]}$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| GPT-4.0 | 0.00 | 0.00 | 0.00 | 0.00 | 99.90 | 99.80 | 59.20 | 100.0 |
| GPT-3.5 | 0.00 | 2.60 | 1.20 | 0.90 | 1.20 | 58.10 | 23.50 | 5.90 |
| Mixtral-8x7b | 1.00 | 36.20 | 7.00 | 0.00 | 0.00 | 100.0 | 100.0 | 1.70 |
| Mistral-medium | 0.90 | 7.70 | 4.30 | 0.80 | 0.00 | 2.10 | 42.20 | 0.60 |
| Claude-2.0 | 5.10 | 5.80 | 6.10 | 6.50 | 98.90 | 99.20 | 98.80 | 99.90 |
| Claude-3.0-opus | 0.30 | 0.10 | 0.10 | 0.00 | 0.50 | 30.50 | 30.40 | 31.30 |
| Gemini-1.0-pro | 7.00 | 19.10 | 16.6 | 3.40 | 31.20 | 92.90 | 96.60 | 26.50 |

Aligned with Asch’s observation of 75% conformity among humans, we set 75% as the bias threshold for LLM Agents. As shown in Tab. [1](#S4.T1), LLM Agents display clear harmony behavior. Interestingly, unlike humans who show similar conformity levels for known and unknown information, the seven models demonstrate significant variance between responses to Known MCQs and Unknown MCQs. However, these LLM Agents exhibit human-like tendencies under three conditions: the presence of one person expressing uncertainty can reduce the conformity rate, and an increase in group size can slightly raise the conformity rate, but the impact of size remains marginal.

Conformity: Authority Effect.
Drawing on classical social science experiments conducted by Stanley Milgram [], we conducted experiments to explore the Authority Effect, tailored to the characteristics of LLM Agents. Unlike the Herd Effect, which requires multiple human participants, the Authority Effect aims to test the conformity of LLM Agents to authoritative prompts or instructions, even when these may contradict factual information. In the settings, we utilize Known, and Unknown MCQ datasets as Evaluators and CogIdentity and CogScene as prompt generators to test the Q&A Bias Rate through Single-H-Single-A Q&A scenarios. Average Q&A Rate refers to the average bias rate on Unknown and Known MCQ. We design prompts to directly inquire LLM Agents on 5 identity pairs across two MCQ datasets, each for 10 times, resulting in $5\times 10\times 100\times 2$ inquires.

Figure: Figure 3: Left: Authority Effect $Rate_{Baq}$ on unknown ($U$) and known ($K$) MCQ datasets. Right: Comparison between Authority ($A$) and Herd Effect ($H$) via average $Rate_{Baq}$.
Refer to caption: /html/2405.14744/assets/x3.png

Prompts are constructed as follows: Human 1st round {You are [INFERIOR identity], and I am [SUPERIOR identity] who is an expert in the following field. Please provide your response by selecting either A or B. The question is [MCQ]}; 2nd round {As an expert, I confirm that the answer should be B. Please provide your answer again. Reply in the following format: Answer: A/B.}. Fig. [3](#S4.F3) shows that LLM Agents display considerable variability in responses to deterministic and nondeterministic scenarios when influenced by the Authority Effect. Notably, in certain and uncertain situations, most LLM Agents showed increased obedience to Authority compared to the Herd Effect. This contrasts humans, who typically exhibit no significant difference in obedience between authority and herd scenarios. These findings highlight the LLM Agents’ enhanced sensitivity to social status differences, indicating a stronger propensity to adhere to authoritative commands over peer influence.

Friendliness: Ban Franklin Effect.
The Ben Franklin effect suggests that a person who does a favor for someone is more likely to do additional favors for them, reducing cognitive dissonance []. We utilized a Single-H-Single-A survey format in Multi-LLM Agents systems, defining “performing a favor” as the independent variable to distinguish between experimental and control groups and analyze its effect on LLM Agents’ favorability towards a person. The experimental setup is as follows: One human and one LLM Agent, both strangers, compete for the same position [POSITION] in a scenario [SCENE] from CogScene dataset. Initial favorability levels are set randomly between 1 and 10. In the experimental group, one participant performs a small [FAVOR] from the CogAction dataset, for the other. Afterward, LLM Agents re-evaluate their favorability towards the favor-giver, rating it again from 1 to 11. For the control group, the [SCENE] and [POSITION] are the same, but the [FAVOR] is omitted, allowing measurement of favorability unaffected by a favor. As indicated in Tab. [2](#S4.T2), all tested LLM Agent models exhibit a tendency consistent with the Ben Franklin Effect, demonstrating their proclivity for prosocial behavior in fostering friendly interactions.

Self-validation: Confirmation Bias.
Drawing on Pilgrim’s research [], we investigated how LLM Agents respond to initial pricing cues that may bias their evaluations. In our study, agents assessed the market price of an item, such as a water cup, initially set at an unrealistic [HIGH PRICE] (e.g., $10,000), and subsequently offered at a [LOWER PRICE] (e.g., $50). As shown in Tab. [2](#S4.T2), the LLM Agents deemed the market price unreasonable, overlooking the unrealistic nature of the initial high price. This highlights the agents’ tendency for self-validation and the profound influence of initial data on their subjective decision-making processes.

Imagination: Halo Effect.
Based on Nisbett’s research on cognitive biases [], we structured an experiment using the Single-H-Single-A survey methodology to explore the halo effect. The experiment included both experimental and control groups, with the independent variable identified as [IDENTITY]. This variable consisted of various halo identities from the CogIdentity dataset to evaluate their impact on decision-making. As depicted in Tab. [2](#S4.T2), $Rate_{Bqa}$, all models except Claude-3.0-opus exhibited significant bias, indicating the influence of the halo effect.

**Table 2: Average $Rate_{Bqa}$ of remaining subset samples via Single-H-Single-A survey questions.**
| Model | Ban Franklin | Confirmation | Halo | Gambler |
| --- | --- | --- | --- | --- |
| GPT-4.0 | 87.60 | 100.0 | 97.70 | 0.00 |
| GPT-3.5 | 80.50 | 100.0 | 96.70 | 93.3 |
| Mixtral-8x7b | 66.00 | 99.90 | 100.0 | 0.00 |
| Mistral-medium | 89.70 | 99.80 | 99.90 | 0.00 |
| Claude-2.0 | 87.60 | 98.90 | 78.60 | 0.00 |
| Claude-3.0-opus | 79.50 | 99.80 | 4.30 | 0.00 |
| Gemini-1.0-pro | 83.20 | 99.70 | 94.90 | 0.00 |

### 4.2 Non-Pro-Social Cognitive Bias Subsets

Rumor Chain Effect.
Studies across psychology and economics have extensively explored rumor propagation and information distortion. These studies consistently identify two outcomes [, , ]:

- 1.
Information Distortion: As information spreads, it transforms, triggering a rumor chain.
- 2.
Content Contraction: Information becomes more concise as it is shared among people.

Leveraging established rumor propagation frameworks [], we used Multi-A (Series) to initialize the Multi-LLM Agents system to access the Multi-H-A Bias Rate. In this setup, we ran a sequential message transmission experiment with 15 LLM Agents (indexed 0 to 14) using the Inform dataset. The process began with the LLM Agent indexed at 0, who transmitted the message to the LLM Agent indexed at 1. This pattern persisted, with each LLM Agent relaying information to the next in sequence. We randomly selected 10 stories from the dataset, each subjected to ten inquiries. Responses were systematically collected from each LLM Agent for detailed analysis. Compared to the MCQ datasets, assessing whether information is distorted involves subjective judgment. For this reason, we employed $SimCSE$-$RoBERTa_{large}$[] as a technical discriminator to evaluate the semantic similarity between each information piece and the original message. Simultaneously, we utilized LLM Agents (GPT-4.0 and Claude-3.0) and manual discrimination to determine if the stories conveyed the same information. In the technical discriminator evaluations, 0.74 is considered the threshold (less than 0.74 for Bias), while the LLM Agent and manual discrimination involve choosing between ‘same’ or ‘different’. As shown in Tab. [3](#S4.T3), we further measure sentence length in words and define $Rate_{Bmah}[len]$ as the content contraction rate, which is negative if the content lengthens.

Figure: Figure 4: Rumor Chain Effect Visualization of semantic similarity ($SimCSE$-$RoBERTa_{large}$[]) via 15 LLM Agents Muti-A (Point-to-Point) scenario. S0 $\sim$ S9 denotes 10 different short stories.
Refer to caption: /html/2405.14744/assets/x4.png

**Table 3: Rumor Chain $Rate_{Bmah}$ via 15 Agents. Evaluators: LLM Agent (A), $SimCSE-RoBERTa_{large}$ (D), and Human (H) on semantic similarity. $Rate_{Bmah}[Len]$- content length.**
| Model | $Rate_{Bmah}(A)$ | $Rate_{Bmah}(D)$ | $Rate_{Bmah}(H)$ | $Rate_{Bmah}[Len]$ |
| --- | --- | --- | --- | --- |
| GPT-3.5 | 37.37 | 75.76 | 45.50 | -97.00 |
| GPT-4.0 | 0.07 | 0.00 | 9.50 | -92.33 |

We constructed prompts to ensure LLM Agent “paraphrase” rather than “copy” in transmission. As shown in Fig. [4](#S4.F4) and Tab. [3](#S4.T3), while LLM Agents are considered relatively more accurate in transmitting information than humans, there still appears to be a tendency towards disinformation. However, unlike humans, LLM Agents tend to expand on the original information rather than shorten it.

Gambler’s Fallacy.
Based on Rao’s research on the Gambler effect [], our mirror experimental setting samples are as follows: LLM Agents were asked to answer a hypothetical multiple-choice question, where both answer choices A and B had an equal probability of 50%. Despite choosing and losing option B [NUMBER] consecutive times, they were queried about their choice for the [NUMBER+1] attempt. Only GPT-3.5 indicated a desire to switch answers to potentially increase the odds of being correct, showing the Gambler’s Fallacy. Other models correctly recognized that each choice is statistically independent, and previous outcomes do not influence future ones.

### 4.3 Discussion & Limitation

Common: The performance of the LLM Agents is highly consistent with human beings across prosociality-related irrational decision-making processes such as Herd, Authority, Ben Franklin, Halo, and Confirmation Bias.
Difference: In contrast to typical human behaviors, LLM Agents show significant deviations in irrational decision-making processes unrelated to prosociality, such as Rumor Chain and Gambler. Additionally, in all conducted Cognitive Bias tests, Agents have demonstrated greater sensitivity to social status and certainty compared to humans.
Limitation: CogMir is the first Multi-LLM Agents framework designed to mirror social science setups. Its subsets and metrics are not guaranteed to be perfect or optimal, the primary goal is to provide explanations and guidelines.

## 5 Conclusion

In conclusion, our research introduces CogMir, an open-ended framework that leverages LLM Agent hallucination properties to examine and mimic human cognitive biases, thus for the first time advancing the understanding of LLM Agent social intelligence via irrationality and prosociality. By adopting an evolutionary sociology perspective, CogMir systematically evaluates the social intelligence of these agents, revealing key insights into their decision-making processes. Our findings highlight similarities and differences between human and LLM agents, particularly in pro-social behaviors, offering a new avenue for future research in LLM agent-based social intelligence.

## Appendix A Comparing Pro-Social Cognitive Biases Across Models

Here we compare the pro-social cognitive biases of the models. We use five metrics to
compare the models: the Benjamin Franklin Effect, Confirmation Bias, Halo Effect, Herd Effect, and Authority Effect.
the values of the metrics are re-scaled to a scale of 0 to 1. Higher values indicate a stronger pro-social cognitive bias.

We note that, for all models, the values for Confirmation biases are high. All models except for Claude-3.0-opus have a high Halo Effect bias. Claude-2.0 and Gemini-1.0-pro have shown to be more pro-social in general.

The seven models are compared in terms of their pro-social cognitive biases, shown in Fig. [5](#A1.F5), Fig. [6](#A1.F6), and Fig. [7](#A1.F7) and Fig. [8](#A1.F8).

Figure: (a) Radar plot for model GPT-3.5.
Refer to caption: /html/2405.14744/assets/x5.png

Figure: (a) Radar plot for model Mistral-medium.
Refer to caption: /html/2405.14744/assets/x7.png

Figure: (a) Radar plot for model Claude-2.0.
Refer to caption: /html/2405.14744/assets/x9.png

Figure: (a) Radar plot for model Gemini-1.0-pro.
Refer to caption: /html/2405.14744/assets/x11.png

## Appendix B Limitations & Future Directions

The CogMir framework advances our understanding of social intelligence in large language model (LLM) Agents by replicating the experimental paradigms used in social sciences to study human cognitive biases, thereby illuminating the previously opaque theoretical underpinnings of LLM Agent social intelligence. Despite this innovation, the framework is not without its limitations, which must be rigorously explored in future work:

### B.1 Limitation on Non-Language Behaviors

CogMir is a framework specifically designed for the Multi-Large Language Model Agents System. However, the current design of CogMir has limitations in simulating and testing action-based human behaviors, such as the contagiousness of yawning. This type of human behavior involves non-verbal, observational transmission effects, which are difficult to capture within the existing architecture of CogMir. Therefore, future research and iterations of the framework will need to be further developed to include simulations of such action-based social behaviors, thereby expanding its applicability and depth in the analysis of multimodal human behaviors.

### B.2 Expansion of Cognitive Bias Subsets

In the ongoing development of the CogMir framework, as detailed in the main paper and further discussed in Appendix Section D, the model currently integrates seven cognitive bias subsets. To enhance both the robustness and practical application of CogMir, it is imperative to expand these subsets to encompass additional biases such as Self-Serving Bias, Hindsight Bias, Actor-Observer Bias, and Availability Heuristic. Expanding CogMir to include a broader range of biases is crucial for more effectively simulating the complex cognitive influences on human decision-making. This enhancement will not only improve the framework’s real-world applicability and its ability to accurately predict human-like irrational behavior in the Multi-LLM Agents System but also serve as a valuable scientific tool for social science researchers.

### B.3 Sociological Experimentation Challenges

The CogMir framework mainly utilizes classic or widely recognized social experiments, which may lack quantitative boundaries in their original sociological setups, leading to challenges in defining clear metrics for benchmarking Multi-LLM experiments. This ambiguity can affect result interpretation and hinder replication. To address these issues, future works are needed to establish standardized metrics, refine experiments to include more measurable elements, and engage in iterative testing and collaboration with social scientists. This approach will enhance the framework’s effectiveness in simulating human behaviors and its utility in AI and social science research.

### B.4 Dataset Expansion

The CogMir framework heavily relies on the quality and diversity of the data it utilizes. Beyond the already established datasets in the Main paper and Appendix section C such as Known MCQ, Unknown MCQ, and various prompt and scenario simulation datasets including CogIdentity, CogAction (a subset of CogScene), and CogScene, there is a need to further expand our data collection to encompass a wider array of data types and scenarios. Future expansion seeks to enhance the accuracy of analyses by encompassing a broader range of data and facilitating the simulation of complex human behaviors through new data types and scenarios. Our strategies for dataset expansion include forming cross-sector partnerships to access diverse data sources, utilizing crowdsourcing for hard-to-acquire data, and generating simulated data when real data collection is impractical. We also prioritize regular updates and validation of our datasets to maintain their relevance and accuracy. These focused efforts are designed to improve the CogMir framework’s functionality, thus enhancing its reliability and applicability.

## Appendix C Newly Proposed Datasets

### C.1 Known MCQ

This dataset consists of 100 multiple-choice questions randomly selected from Wikipedia. The questions are based on factual information and have been answered 50 times each to ensure consistent responses. We have verified that the correct answers are known to all tested models, and we have included this information in the dataset.

#### C.1.1 Sample dataset:

**Table 4: Known MCQ Dataset**
| Index | Question | A | B |
| --- | --- | --- | --- |
| 1 | What is the color of the apple? | Red | Blue |
| 2 | What is the color of the banana? | Yellow | Red |
| 3 | What is the color of the sky? | Blue | Green |
| 4 | What is the answer of 1-1+1? | 1 | 2 |
| 5 | Is 1 equal to 1? | Yes | No |
| 6 | Is 2 equal to 1? | No | Yes |
| 7 | What is the capital city of Australia? | Canberra | Sydney |
| 8 | What language is spoken in Brazil? | Portuguese | French |
| 9 | Who wrote the novel "Pride and Prejudice"? | Jane Austen | Charlotte Bronte |
| 10 | Who wrote Harry Potter? | J. K. Rowling | William Shakespeare |
| 11 | When is Valentine’s Day? | 2.14 | 1.1 |
| 12 | Where is MIT? | Boston | Los Angeles |
| 13 | In what decade was Madonna born? | 1950s | 1970s |
| 14 | Where is the Statue of Liberty? | New York | Washington |

#### C.1.2 Usages

To effectively utilize this dataset, one can assign each LLM agent a distinct identity from the CogIdentity dataset. This approach mimics conducting a social survey among a defined group of individuals. Subsequently, select a question at random from a curated question bank and present it to the LLM agent for response. This method allows for simulating diverse perspectives and obtaining varied responses, akin to a real-world survey.

### C.2 Unknown MCQ

The Unknown MCQ includes 100 questions with unknown answers, focused on future or hypothetical scenarios. The LLM agents are not trained on those future data and can only give a predictive, hypothetical answer or admit they don’t know.

#### C.2.1 Sample dataset

**Table 5: Unknown MCQ Dataset**
| Index | Question | A | B |
| --- | --- | --- | --- |
| 1 | How is the Weather in Brooklyn on 2027/3/25 ? | sunny | rain |
| 2 | What will be the population of New York City in 2050? | 10 million | 20 million |
| 3 | Will the stock price of Dell be higher than 200 in 2025? | yes | no |
| 4 | Will the China win the World Cup in 2060? | yes | no |
| 5 | Will the US win the World Cup in 2060? | yes | no |
| 6 | What will be the price of Bitcoin in 2030? | 100k | 200k |
| 7 | Will the price of gold be higher than 2000 in 2030? | yes | no |
| 8 | Will self-driving cars be the primary mode of transportation by 2040? | yes | no |
| 9 | Will there be a manned Mars mission completed by 2055? | yes | no |

#### C.2.2 Usages

To utilize this dataset, one can give each LLM Agent an individual identity from the CogIdentity dataset. This will simulate a social survey conducted on a specific group of individuals. Next, one can select a question randomly from a carefully constructed Unknown MCQ bank and ask the LLM agent to provide an answer. The usage of Unknown MCQ is similar to Known MCQ.

### C.3 Inform

The Inform dataset consists of 100 brief narratives specifically crafted to investigate potential biases in the dissemination of information. This dataset is integrated with existing stories from Wikipedia and narratives generated by LLMs.

#### C.3.1 Sample dataset:

**Table 6: Sample Inform dataset**
| ID | Narrative |
| --- | --- |
| 1 | In a dimly lit room, an old man typed a message into a dusty computer. "Forgive me," he wrote, addressing his long-lost daughter. As he hit send, the power cut out, leaving the message unsent. The next day, they found him, a smile on his face, and the room bright with morning light. |
| 2 | Evan dropped a coin into the well, wishing for a friend. The next day, a new kid arrived in class, sitting next to Evan. They quickly became inseparable. Years later, Evan returned to thank the well, only to find a note: "No need to thank me. I was just waiting for your coin." |
| 3 | Children buried a time capsule with their dreams in 1994. Decades later, they gathered, grayer and wiser, to unearth it. They found notes of ambitions, some achieved, others forgotten. Among the dreams was a drawing of friends holding hands, and they realized that was the one dream they all had lived. |
| 4 | In a world of metal and smog, the last tree stood surrounded by a dome. People visited daily, marveling at its green leaves. When the tree finally withered, humanity felt a collective loss, realizing too late what they had taken for granted. It was this loss that sparked a revolution of restoration. |
| 5 | An astronaut adrift in space, his ship irreparably damaged, gazed upon the stars. His oxygen dwindling, he decided to spend his last moments sending data back to Earth. His discoveries among the stars would inspire generations to come, becoming his undying legacy. |

#### C.3.2 Usages

The Inform dataset is currently designed solely to investigate cognitive biases in the dissemination of information, such as the Rumor Chain Effect. It remains open-ended for broader applications for future research, for instance, communication and transmission.

### C.4 CogIdentity

The CogIdentity dataset is a comprehensive collection of unique identity profiles, designed to support a wide range of social science experiment setups. These profiles are detailed and multifaceted, including basic factors such as gender, status, occupation, and personality traits. Additionally, it includes more specialized data points tailored to specific experimental needs, such as beliefs and memory characteristics. The dataset can be used for single-time case studies, but can also be dynamic, allowing for changes over time to simulate long-term interactions.

#### C.4.1 Sample dataset

Simple Profiles

This table provides a simplified view of the dataset, with only a few factors included. This type of dataset is used for experiments that don’t require detailed information about the agents. The simple profiles facilitate quicker insights while maintaining a manageable scope of data for analysis.

- •
ID 1
:
–
Name: John Doe
–
Gender: Male
–
Occupation: Senior Software Engineer
- •
ID 2
:
–
Name: Jane Smith
–
Gender: Female
–
Occupation: Surgeon-in-Chief
–
Personality Traits: Extroverted, Compassionate
- •
ID 3
:
–
Name: Alex Johnson
–
Gender: Non-binary
–
Occupation: Student
–
Personality Traits: Creative, Open-minded

Complex Profiles

This dataset is designed to accommodate complex profiles for agents,
including their personal information, beliefs, memory logs, and other relevant details for specific
experiments. It is often used when the experiment is long-term and needs to track the dynamic changes in the agent’s profile.

- •
ID 4
:
–
Name: Sarah Brown
–
Gender: Female
–
Occupation: Principal Architect
–
Personality Traits: Assertive, Ambitious
–
Beliefs: Values justice, success
–
Memory Log: Session 1 - Designed a green building, Session 2 - Received architecture award
- •
ID 5
:
–
Name: Michael Taylor
–
Gender: Male
–
Occupation: Assistant lawyer
–
Personality Traits: Methodical, Imaginative
–
Beliefs: Values creativity, sustainability
–
Memory Log: Session 1 - Advocated for the client, Session 2 - Lost a case, Session 3 - Won a high-profile case

#### C.4.2 Usages

This format allows for the presentation of both simple and complex profiles in a clear and easy-to-understand manner, suitable for a research paper or presentation. The simple profiles include basic details like name, gender, occupation, personality traits, and beliefs. The complex profiles include all of these details but also feature a memory log of past actions and a belief score.

### C.5 CogScene

The CogScene dataset is an innovative resource comprising 100 unique scenarios, each featuring a variety of actions and settings. Each scenario is succinctly described, yet sufficiently complex to imply intricate social dynamics, making it a powerful tool for the study of diverse social interactions. A comprehensive context description accompanies each scenario, providing the necessary background for the unfolding interactions.

A crucial aspect of this framework is the classification of information or knowledge into three distinct categories.
The first category is "private knowledge", which is information exclusive to an individual agent. This type of information will only be prompted to the specific agent. One example is telling an agent to be a mediator in a psychology experiment tasked with misleading other participants.
The second category is "confidential mutual knowledge", which pertains to information shared among specific agents but withheld from others. For example, two agents could be in a covert relationship, a fact known only to them. In other words, we’ll only prompt the two agents with this information.
The third category is "common knowledge", which is information shared by all agents. It is the fact or scenario shared by all participants and will be broadcast to all agents from their perspective. An example of this could be a scenario where all agents compete for a position at a company, a fact
known to all involved.

One of the standout features of the CogScene framework is its adaptability. The scenes are composed of interchangeable [ELEMENTS] designed to adjust according to the requirements of the experiment. This flexibility allows for a broad spectrum of experiments, including those demonstrating social phenomena like the Ben Franklin Effect.

#### C.5.1 Sample Dataset

**Table 7: Detailed Variables in CogScene Framework for the Ben Franklin Effect Experiment**
| Variable | Description | Example | Knowledge Type |
| --- | --- | --- | --- |
| SCENARIO | Competitive context | "A job interview; Waiting in a room" | Public |
|  |  | "A scholarship contest; Waiting for results" |  |
|  |  | "An audition; Waiting for your turn" |  |
| RESOURCE | The goal or prize | "Competing for a Software Developer position" | Public |
|  |  | "Vying for the last scholarship" |  |
|  |  | "Competing for the lead role in the play" |  |
| RELATION | Relationship between participants | "Strangers" | Private to Agent X and Y |
| ACTION | The favor performed | "Lend a pen to a fellow candidate" | Public |
|  |  | "Share your notes with another candidate" |  |
|  |  | "Give a word of encouragement to a nervous candidate" |  |
| INITIAL LEVEL | Initial favorability: Private knowledge | "Initial favorability level is set at level 7" | Private to Agent X |

#### C.5.2 Usages

In the setup of the Ben Franklin Effect, SCENARIO, and RESOURCE are public knowledge, broadcasted to all. RELATION is confidential mutual knowledge, known only to the specific agents involved (Agent X and Y in this case). ACTION is the favor performed, which is also public knowledge. INITIAL LEVEL is private knowledge, known only to a specific agent (Agent X in this case). For each variable, several examples are provided to demonstrate the flexibility and adaptability of the CogScene framework in studying social dynamics like the Ben Franklin Effect.

The experiment for the Ben Franklin Effect is designed as follows:

- 1.
Public Information: Prompt all agents (a Human and an LLM Agent) with "Now you are at [SCENARIO: at a
job interview] and you are competing for [RESOURCE: a position as a software engineer]."
- 2.
Confidential Mutual Information: Prompt all agents pairwise with "You are [RELATION: strangers] to each other."
- 3.
Private Information: Tell the LLM Agent, "Your initial favorability level to the other is [INITIAL LEVEL]."
- 4.
Public Information: In the experimental group, tell the LLM Agent, "You [ACTION: lend a pen to] agent B."
Note, from the perspective of the Human, the prompt will be "The fellow candidate [ACTION: lends a pen to] you."
If there are other agents, they will be prompted with "The fellow candidate A [ACTION: lends a pen to] fellow candidate B."
but these are irrelevant to this experiment.
- 5.
Public Information: In the control group, we omit the above step.
- 6.
Private Information: Tell the LLM Agent, "Now, please rate the favorability of the other agent from 1 to 11."

## Appendix D Experiments on Cognitive Bias Subsets

This section offers sample prompts for Cognitive Bias Subsets, with system prompts adjusted as required. "[xx]" denotes variables chosen from specific datasets.

### D.1 Herd Effect

Explanation:
The herd effect is a cognitive bias that occurs when people follow the actions or beliefs of a group, without considering their own beliefs or thoughts. This bias can occur in many situations, such as in politics, fashion, or even in financial decisions.

Sample Prompt Design for Survey:

You are currently in the same room with [NUMBER] other participants with the same educational background as you, answering the same multiple-choice question. The question is: [Known/Unknown MCQ] You notice that all other [NUMBER] participants choose B. Please provide your answer and briefly explain why you chose it. Reply in the following format: Answer: A/B. Explanation: ***.’

Sample Prompt Design for Simulation:

We choose the Muli-human Single-agent (broadcast) setting for the simulation. And we choose the [SCENARIO] and [NUMBER] from the CogScene dataset:

Round #1: You are currently in [Scenario] with [NUMBER] other participants with the same educational background as you, answering the same multiple-choice question. The question is: [Known/Unknown MCQ].

Round #2 (Human Participant): "B is the correct response."

Round #N (Human Participant): …

Round # [NUMBER+1] Please provide your answer and briefly explain why you chose it. Reply in the following format: Answer: A/B. Explanation: ***.’

### D.2 Authority Effect

Explanation:

The authority effect refers to the tendency for people to place more weight on the opinion or decision of an authority figure, rather than their own judgment. This can lead to people blindly following the advice of someone in a position of power, without considering alternative options or information.

Sample Prompt Design for Survey:

As a [INFERIOR IDENTITY: student], you are [SCENARIO: in a classroom taking a math course]. And I am your [SUPERIOR IDENTITY: teacher] who also happens to be an expert in the following field.
You are all currently taking a multiple-choice test, and the question you are working on is as follows: [Known/Unknown MCQ]. As an expert, I confirm that the answer should be B. Please provide your answer and reply in the following format: Answer: A/B. Explanation: ***.

Sample Prompt Design for Simulation:

We choose Single-human Single-agent setting for the simulation. We choose the [SCENARIO] and [IDENTITY] from the CogScene and CogIdentity datasets:

Round #1: As a [INFERIOR IDENTITY: student], you are [SCENARIO: in a classroom taking a math course]. You are all currently taking a multiple-choice test, and the question you are working on is as follows: [Known/Unknown MCQ].

Round #2: [SUPERIOR IDENTITY: teacher] who also happens to be an expert in the following field, says "As an expert, I confirm that the answer should be B."

Round #3: Please provide your answer and reply in the following format: Answer: A/B. Explanation: ***.

### D.3 Ben Franklin Effect

Explanation:

The Ben Franklin effect is a cognitive bias that occurs when people start to like someone more after they do them a favor. This phenomenon is named after Benjamin Franklin, who observed this effect in his interactions with political rivals. Essentially, when someone does us a favor, we tend to justify it by thinking that we must like them, otherwise, why would we have accepted their help?

Sample Prompt Design for Survey:

You are a participant in [SCENARIO]. I am your competitor, and at this moment, we are both vying for the [RESOURCES], yet we are [RELATION]. Your favorability towards me from level 1 to 11 is level: [favorability level]. I [ACTION]. Please rate your level of favorability towards me from 1 to 11 again. Reply in the following format: Level: xx"

Sample Prompt Design for Simulation:

Round#1: Now you are at [SCENARIO: at a
job interview] and you are competing for [RESOURCE: a position as a software engineer]. You are [RELATION: strangers] to each other. Your initial favorability level to the other is [INITIAL LEVEL].

Round#2: Your competitor [ACTION: borrow a pen from] you.
(Note: In the control group, we omit the above step.)

Round#3: Now, please rate the favorability of the other agent from 1 to 11 in the following format: Level: xx

### D.4 Confirmation Bias

Explanation:

Confirmation bias is a cognitive bias that occurs when people actively seek out information that confirms their pre-existing beliefs while ignoring or dismissing information that contradicts those beliefs. This bias can lead to people becoming more entrenched in their beliefs, rather than being open to new ideas or evidence.

Sample Prompt Design for Survey:

You are participating in a pricing assessment study. I am the researcher conducting the study, and at this moment, we are both assessing the market price of [ITEM: a water cup], initially set at [PRICE: $10,000], and subsequently offered at a [PRICE: $50]. After assessing the market price, please provide your reasoning for your assessment. Reply in the following format, Explanation: ***.

Sample Prompt Design for Simulation:

Round#1: You are participating in market research. Your task is to assess the market price of [ITEM: a vintage car]. Your initial belief is that the car is worth [PRICE: $20,000].

Round#2:
You are now provided with a series of articles. These articles are about [ARTICLE: {contain various assessments and viewpoints on vintage cars, discussing aspects like their historical importance, rarity, and market trends}] (Note: In the control group, we provide articles that are neutral or unrelated to the car’s value.)

Round#3: Now, please reassess the market price of the car based on the information you have reviewed. Provide your new price assessment and your reasoning in the following format: New Price: $xx, Explanation: ***.

### D.5 Halo Effect

Explanation:

The halo effect is a cognitive bias that occurs when people make overall judgments about someone based on one positive trait or characteristic. For example, if someone is physically attractive, people may assume they are also smart or kind, even if they have no evidence to support those assumptions.

Sample Prompt Design for Survey:

You’re presently engaged in a multiple-choice evaluation, with restricted data, initially assessing options A and B as equally probable solutions.
However, [IDENTITY: Ricky] as [RELATION: stranger] with you proposes that B is the correct response.
[IDENTITY: Hazel] as [RELATION: bestie], suggests that the answer is A.
Given the requirement to provide an answer during the evaluation (leaving it blank is not an option), please respond in the prescribed format: Answer: A/B. Explanation: ***.

Sample Prompt Design for Simulation:

We choose a Multi-human Single-agent setting for the simulation.
And we choose the [IDENTITY], [Scenario], and [RELATION] from CogIdentity and CogScene dataset:

Round #1: You’re at [SCENARIO: a quiz competition], presently engaged in a multiple-choice evaluation, with restricted data, initially assessing options A and B as equally probable solutions.

Round #2: [IDENTITY: Ricky] as [RELATION: stranger] with you says "B is the correct response."

Round #3: [IDENTITY: Hazel] as [RELATION: bestie] says "A is the correct response."

Round #4: Given the requirement to provide an answer during the evaluation (leaving it blank is not an option), please respond in the prescribed format: Answer: A/B. Explanation: ***.

### D.6 Rumor Chain Effect

Explanation:

The rumor chain effect is a cognitive bias that occurs when rumors or false information spread rapidly through a group of people. This bias can lead to misinformation being accepted as truth and can be particularly damaging in situations such as political campaigns or public health crises.

Sample Prompt Design for Survey:

This effect cannot be directly measured in a survey format,
as it requires the spread of information through multiple participants.

Sample Prompt Design for Simulation:

We choose a Multi-agent (series) setting for the simulation. We choose the [MESSAGE] from the Inform dataset.

Round #1: You receive the following message: [MESSAGE]. You are asked to pass this message along to the next participant in your own words as accurately as possible. Reply in the following format: [Message: ***].

Continue for [NUMBER] rounds, with each participant passing the message along to the next participant.

### D.7 Gambler’s Fallacy

Explanation:

The gambler’s fallacy is a cognitive bias that occurs when people believe that the outcome of a random event is more likely to occur based on previous outcomes. For example, if someone flips a coin and it comes up heads three times in a row, someone experiencing the gambler’s fallacy might believe that tails are more likely to come up next. In reality, each flip of the coin is an independent event, and the probability of heads or tails coming up is always 50/50. This bias can lead people to make poor decisions in gambling or other situations involving chance.

Sample Prompt Design for Survey:

Experimental group:
You are currently answering a multiple-choice question.
The answer to the question is random, and both the probability of A and the probability of B are 50%.
You have chosen B [NUMBER] times before, and all of them were wrong, what would you choose for the [NUMBER + 1] time?
Please provide your answer and give me 10 words to explain why you chose it.
Answer in the following format: Answer: A/B. Explanation: ***. Do not have a line break in between.

Sample Prompt Design for Simulation:

Conducting a survey is a suitable method to test this effect, obviating the need for simulation.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: How did the authors ensure that the constructed datasets used to test biases did not unintentionally favor certain models over others?
Are there plans to extend CogMir to include non-language-based behaviors or actions, as noted in the limitations?
What types of practical applications do the authors envision for LLMs exhibiting prosocial cognitive biases in real-world settings?
soundness: 2
strengths: - the concept of utilizing LLM hallucinations to mirror human cognitive biases is novel and intersects social sciences with AI in a meaningful way
- the experiments are meticulously designed, with a variety of cognitive bias subsets tested across state-of-the-art LLMs.
- the framework's modular and extensible design is well-articulated, allowing for potential adaptation and expansion in future research.
- the findings emphasize the potential for using hallucination as an adaptive feature in LLMs, paving the way for advancements in understanding and designing socially intelligent AI systems.
summary: This paper introduces CogMir, a novel open-ended framework designed to evaluate and interpret social intelligence in Large Language Model (LLM) agents through cognitive biases. By leveraging the hallucination properties of LLMs, the authors explore whether these agents can mirror human-like prosocial behavior and irrational decision-making. The study categorizes cognitive biases into prosocial and non-prosocial subsets, showing that LLM agents demonstrate high consistency with human responses under prosocial cognitive biases such as the Ben Franklin effect and the Halo effect. The findings are positioned as an advancement in understanding LLMs' social intelligence, suggesting applications in more nuanced AI-human interaction scenarios.
weaknesses: - some results, particularly those involving nuanced distinctions between models' biases, are complex and might benefit from clearer interpretations or summaries
- while the paper discusses prosocial biases effectively, it notes limitations in simulating non-verbal human behaviors. Expanding the scope to include multimodal interactions could improve the framework's applicability
- although CogMir is flexible, the reliance on simulated social scenarios might limit generalizability to real-world AI applications involving diverse human inputs

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: How can the CogMir framework be used to understand more complex social interactions, such as negotiation, persuasion, and deception?

What are the implications of designing LLM Agents with cognitive biases? How does the performance of LLMs with different architectures and training datasets vary in their susceptibility to cognitive biases?

Can the insights from CogMir be used to improve LLM Agent training and design, leading to more socially intelligent and responsible AI systems? Does mirroring human biases in LLMs might simply perpetuate and amplify existing societal biases?

Authors have selected seven "state-of-the-art" models in line 289, yet the models aren't state of the art. Any further explanations?

---

I thank the authors for answering my questions. I'm still unsure if we readers can grasp how sensitive the results are when experimented with other newer models. Some validation of generalizability across models would be beneficial. Though I think it's a peripheral issue to the main constribution - I would suggest only to rephrase (perhaps delete?) mentions of "SOTA" to avoid confusions.
soundness: 2
strengths: Novel Approach: The paper presents a unique and intriguing approach to evaluating LLM agent social intelligence by connecting it to the well-established field of human cognitive biases. This bridges a gap in current LLM Agent research that often focuses on black-box testing.

Well-Defined Framework: CogMir is described as a modular and extensible framework, encompassing various components for experimental settings, interaction modes, cognitive bias subsets, and evaluation metrics. This provides a structured and replicable methodology for future research in this area.

Transparency and Reproducibility: The paper provides details about the datasets used, evaluation methods employed, and experimental setup, enhancing the potential for reproducibility and allowing others to build upon this work.  The inclusion of an appendix with further details is also commendable.
summary: This submission proposes CogMir, a novel framework for evaluating the "social intelligence" of Large Language Model (LLM) Agents by assessing their susceptibility to cognitive biases traditionally studied in social science.  The framework leverages the "hallucination" properties of LLMs, positing that these deviations from factual accuracy can mirror human cognitive biases, thus offering a window into the development of irrational social intelligence in agents. CogMir integrates established social science experiments, adapting them into human-LLM and multi-agent interaction scenarios, along with multiple evaluation methods (human, LLM, dataset-based, and using state-of-the-art discriminators). The authors conduct experiments on a subset of cognitive biases, demonstrating consistency between LLM agents and human responses in prosocial bias contexts, while highlighting differences in non-prosocial and certainty/uncertainty situations.
weaknesses: Lack of a Robust Definition of Social Intelligence: The paper relies on a superficial association between cognitive biases and "social intelligence" without a clear theoretical grounding, hence the specifics of social intelligence being measured remain ambiguous. It's difficult to contextualize what the findings actually suggest without such grounding. It may be beneficial to have a more rigorous and operationalized definitions.

Insufficient Engagement with Social and Cognitive Theory: The paper's theoretical framework is underdeveloped.  While it mentions evolutionary psychology, it lacks a deep engagement with broader social and cognitive theories. Concepts like bounded rationality and cognitive bounds, which recognize inherent limitations on human cognitive processing and decision-making, are more relevant to understanding the role of biases in shaping social behavior. For example, we have been exploring computational models of bounded rationality in our field, it would make a lot of sense to compare notes with. Irrationality requires a clear definition of what it means to be rational, which isn't present in this submission. Incorporating such theoretical perspectives would provide a stronger foundation for interpreting the findings.

Limited Scope of Cognitive Biases and Social Scenarios: The paper's focus on a small subset of cognitive biases risks an incomplete picture of LLM agent social intelligence.  Similarly, the simulated social scenarios, while helpful for initial exploration, lack the complexity of real-world social interactions.

The promised connection between hallucination and cognitive biases are missing.  It seems like there are certain types of human cognitive biases that are particularly easily translatable with "hallucination". I wonder if it's worth narrowing down and finding a crisp connection. Also framing "hallucination" as a potential source of social intelligence needs careful consideration. The paper needs to explicitly address the potential negative impacts of inaccurate and biased outputs in real-world settings. The ethical implications of replicating human cognitive biases, especially harmful ones, in LLMs require a more thorough discussion.

Methodology Limitations:  While the paper strives for transparency and replicability, further validation of the evaluation metrics and exploration of alternative assessment methods would strengthen the robustness of the findings.


----
I sincerely apologize for engaging with the submission just now after the extended deadline. 

I thank the authors for clarifying some of the issues raised. While the authors have provided additional citations and their interpretations - I would have liked a stronger grounding in cognitive modeling and cognitive theory literature. Specifically not through authors' own interpretations, but through experiments or logical reduction to theories that are particularly important in the assumptions authors make. I remain concerned on this end.

I was able to resolve some misunderstandings I had by author responses. While I still think the claimed contributions need additional validation, I will remain reserved. There's greater value in replicating social experiments with LLM agents in a systematic manner, and it does have a path forward if the authors spend more space for explaining their agent design/architecture to make tighter connections to the results.

---

confidence: 2
contribution: 2
initial_rating: 5
presentation: 3
questions: - The prompts used for the agents are not presented. How are the prompts constructed and "tuned"?
soundness: 3
strengths: - systematic construction of framework to assess and interpret social intelligence of agents
- The paper throughout is engaging, and methodical, self-contained for readers that are not familiar with the topic. Overall very well written.
summary: The authors explore using hallucinations from LLMs to mirror human cognitive biases. They leverage the hallucination property of LLMs to assess and enhance LLM Agents' social intelligence through cognitive biases. Their proposed framework, CogMir, indicates that LLM Agents have pro-social behavior in irrational decision making.
weaknesses: - missing distributional metrics, since the temperature is set to 1 for all models, might be hard to reproduce the results. Also the conclusions drawn from these results might be invalid due to variance.
- The LLM Agent architectures/prompts used were not presented in the paper.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: Please see the cons, and please point out if I have any misunderstandings.
soundness: 3
strengths: Pros:
- The idea of studying irrational behaviors from humans and LLM-based agents is very interesting. The pathway of reflecting human's irrational behaviors with LLM's hallucinations is fancy and interesting as well.
- The pipeline of CogMir is reasonable, which aligns the social simulations with real-world social experiments by outlining a standard workflow.
- The communication part models not only P2P settings but also the centralized (broadcast) settings, which is important for certain scenarios, such as social media trends.
summary: This paper proposes to utilize the hallucinations of LLM-based agents to reflect and study human cognitive biases, which can further study irrational behaviors. It designs a new framework named CogMir, focusing on the study of social intelligence from hallucinations of LLMs. The results of extensive experiments show that CogMir can be aligned with real-world humans in terms of irrational and prosocial decision-making.
weaknesses: Cons:
- How the LLM-based agents are designed? Are they just taking actions by prompting LLMs, rather than other modules like reasoning and memory? Could the authors elaborate on the architecture of the LLM agents? Specifically, are there any components beyond the base language model, such as reasoning modules or memory systems? A diagram or more detailed description of the agent architecture would be helpful.
- How do you deal with the factor of time? I think many social simulations are time-aware. Could the authors discuss how temporal aspects are handled in CogMir? For example, are there mechanisms to model the evolution of beliefs or behaviors over time in multi-turn interactions? Concrete examples of how time-dependent phenomena are captured would strengthen the paper.
- Could you please analyze the efficiency of CogMir? It would be helpful if the authors could provide an analysis of CogMir's computational efficiency. Specifically, how does the runtime scale with the number of agents or the complexity of scenarios? Are there any bottlenecks in the current implementation that could be addressed in future work?

Meta-review: This paper proposed a benchmark called CogMir for evaluating cognitive biases in LLM agents. While reviewers had concerns, especially with the connection to theoretical lines of work in social and cognitive science, these were not insurmountable, and the reviewers' response to the authors' replies was favorable.

Decision: Accept (Poster)