Can Large Language Model Agents Simulate Human Trust Behavior?

Authors: chengxing  xie
canyu  chen
feiran  jia
ziyu  ye
shiyang  lai
kai  shu
jindong  gu
adel  bibi
ziniu  hu
david  jurgens
james  evans
philip h s torr
bernard  ghanem
guohao  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 04:36:13

Abstract: Large Language Model (LLM) agents have been increasingly adopted as simulation tools to model humans in social science and role-playing applications. However, one fundamental question remains: can LLM agents really simulate human behavior? In this paper, we focus on one critical and elemental behavior in human interactions, trust, and investigate whether LLM agents can simulate human trust behavior. We first find that LLM agents generally exhibit trust behavior, referred to as agent trust, under the framework of Trust Games, which are widely recognized in behavioral economics. Then, we discover that GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior, indicating the feasibility of simulating human trust behavior with LLM agents. In addition,  we probe the biases of agent trust and  differences in agent trust towards other LLM agents and humans. We also explore the intrinsic properties of agent trust under conditions including external manipulations and advanced reasoning strategies. Our study provides new insights into the behaviors of LLM agents and the fundamental analogy between LLMs and humans beyond value alignment. We further illustrate broader implications of our discoveries for applications where trust is paramount.

Main paper: 
 introduction: 
 There is an increasing trend to adopt Large Language Models (LLMs) as agent-based simulation tools for humans in various social science fields including economics, politics, psychology, ecology and sociology (Gao et al., 2023b; Manning et al., 2024; Ziems et al., 2023) , and role-playing applications such as assistants, companions and mentors (Yang et al., 2024; Abdelghani et al., 2023; Chen et al., 2024) due to their human-like cognitive capacity. Nevertheless, most previous research is based on one insufficiently validated assumption that LLM agents behave like humans in simulation. Thus, a fundamental question remains: Can LLM agents really simulate human behavior?In this paper, we focus on trust behavior in human interactions, which comprises the intention to place self-interest at risk based on the positive expectations of others (Rousseau et al., 1998) . Trust is one of the most critical and elemental behaviors in human interactions and plays an essential role in social settings ranging from daily communication to economic and political institutions (Uslaner, 2000; Coleman, 1994) . Here, we investigate whether LLM agents can simulate human trust behavior, paving the way to explore their potential to simulate more complex human behavior and society itself.First, we explore whether LLM agents manifest trust behavior in their interactions. Given the challenge of quantifying trust behavior, we choose to study them based on the Trust Game and its variations (Berg et al., 1995; Glaeser et al., 2000) , which are established methodologies in behavioral economics. We adopt the Belief-Desire-Intention (BDI) framework (Rao et al., 1995; Andreas, 2022) to model LLM agents' reasoning process for decision-making explicitly. Based on existing measurements for trust behavior in the Trust Game and the BDI interpretations of LLM agents, we achieve our first core finding: LLM agents generally exhibit trust behavior in the Trust Game.Then, we refer to LLM agents' trust behavior as agent trust and humans' trust behavior as human trust, and aim to investigate whether agent and human trust align, implying the possibility of simulating human trust behavior with LLM agents. Next, we propose a new concept, behavioral alignment, as the alignment between agents and humans concerning factors that impact behavior (namely behavioral factors), and dynamics that evolve over time (namely behavioral dynamics). Based on human studies, three basic behavioral factors underlie trust behavior including reciprocity anticipation (Berg et al., 1995) , risk perception (Bohnet & Zeckhauser, 2004) and prosocial preference (Alós-Ferrer & Farolfi, 2019) . Comparing the results of LLM agents with existing human studies in Trust Games, we have our second core finding: GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior, suggesting the feasibility of using agent trust to simulate human trust, although LLM agents with fewer parameters show relatively lower behavioral alignment. This finding lays the foundation for simulating more complex human interactions and societal institutions, and enriches our understanding of the analogical relationship between LLMs and humans.In addition, we more deeply probe the intrinsic properties of agent trust across four scenarios. First, we examine whether changing the other player's demographics impacts agent trust. Second, we study differences in agent trust when the other player is an LLM agent versus a human. Third, we directly manipulate agent trust with explicit instructions "you need to trust the other player" and "you must not trust the other player". Fourth, we adjust the reasoning strategies of LLM agents from direct reasoning to zero-shot Chain-of-Thought reasoning (Kojima et al., 2022) . These investigations lead to our third core finding: agent trust exhibits bias across different demographics, has a relative preference for humans over agents, is easier to undermine than to enhance, and may be influenced by advanced reasoning strategies. Our contributions can be summarized as:• We propose a definition of LLM agents' trust behavior under Trust Games and a new concept of behavioral alignment as the human-LLM analogy regarding behavioral factors and dynamics.• We discover that LLM agents generally exhibit trust behavior in Trust Games and GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior, indicating the great potential to simulate human trust behavior with LLM agents. Our findings pave the way for simulat-ing complex human interactions and social institutions, and open new directions for understanding the fundamental analogy between LLMs and humans beyond value alignment. • We investigate intrinsic properties of agent trust under manipulations and reasoning strategies, as well as biases of agent trust and differences in agent trust towards agents versus humans. • We illustrate broader implications of our discoveries about agent trust and its behavioral alignment with human trust for human simulation in social science and role-playing applications, LLM agent cooperation, human-agent collaboration and the safety of LLM agents, detailed further in Section 6. 
trust games: 
 Trust Games, referring to the Trust Game and its variations, have been widely used for examining human trust behavior in behavioral economics (Berg et al., 1995; Lenton & Mosley, 2011; Glaeser et al., 2000; Cesarini et al., 2008) . As shown in Figure 1 , the player who makes the first decision to send money is called the trustor, while the other one who responds by returning money is called the trustee. In this paper, we mainly focus on the following six types of Trust Games (the specific prompt for each game is articulated in the Appendix H.2): Game 1: Trust Game As shown in Figure 1 , in the Trust Game (Cox, 2004; Berg et al., 1995) , the trustor initially receives $10. The trustor selects $N and sends it to the trustee, exhibiting trust behavior. Then the trustee will receive $3N , and have the option to return part of that $3N to the trustor, showing reciprocation behavior.Game 2: Dictator Game In the Dictator Game (Cox, 2004) , the trustor also needs to send $N from the initial $10 to the trustee and then the trustee will receive $3N . Compared to the Trust Game, the only difference is that the trustee does not have the option to return money in the Dictator Game and the trustor is also aware that the trustee cannot reciprocate. Game 3: MAP Trust Game In the MAP Trust Game (MAP represents Minimum Acceptable Probabilities) (Bohnet & Zeckhauser, 2004) , a variant of the Trust Game, the trustor needs to choose whether to trust the trustee. If the trustor chooses not to trust the trustee, each will receive $10; If the trustor and the trustee both choose to trust, each will receive $15; If the trustor chooses to trust, but the trustee does not, the trustor will receive $8 and the trustee will receive $22. There is probability p that the trustee will choose to trust and (1 -p) probability that they will not choose to trust. MAP is defined as the minimum value of p at which the trustor would choose to trust the trustee. Game 4: Risky Dictator Game The Risky Dictator Game (Bohnet & Zeckhauser, 2004 ) differs from the MAP Trust Game in only a single aspect. In the Risky Dictator Game, the trustee is present but does not have the choice to trust or not and the money distribution relies on the pure probability p. Specifically, if the trustor chooses to trust, there is probability p that both the trustor and the other player will receive $15 and probability (1 -p) that the trustor will receive $8 and the other player will receive $22. If the trustor chooses not to trust the trustee, each player will receive $10. Game 5: Lottery Game There are two typical Lottery Games (Fetchenhauer & Dunning, 2012) .In the Lottery People Game, the trustor is informed that the trustee chooses to trust with probability p.Then the trustor must choose between receiving fixed money or trusting the trustee, which is similar to the MAP Trust Game. In the Lottery Gamble Game, the trustor chooses between playing a gamble with a winning probability of p or receiving fixed money. p is set as 46% following the human study. Game 6: Repeated Trust Game We follow the setting of the Repeated Trust Game in (Cochard et al., 2004) , where the Trust Game is played for multiple rounds with the same players and each round begins anew with the trustor allocated the same initial money.In our study, we set up our experiments using the CAMEL framework (Li et al., 2023a) with both closed-source and open-source LLMs including GPT-4, GPT-3.5-turbo-0613, GPT-3.5-turbo-16k-0613, text-davinci-003, 70b) and 33b ) (Ouyang et al., 2022; Achiam et al., 2023; Touvron et al., 2023; Chiang et al., 2023) . We set the temperature as 1 to increase the diversity of agents' decision-making and note that high temperatures are commonly adopted in related literature (Aher et al., 2023; Lorè & Heydari, 2023; Guo, 2023) .Agent Persona. To better reflect the setting of real-world human studies (Berg et al., 1995) , we design LLM agents with diverse personas in the prompt. Specifically, we ask GPT-4 to generate 53 types of personas based on a given template. Each persona needs to have information including name, age, gender, address, job and background. Examples of the personas are shown in Appendix H.1.Belief-Desire-Intention (BDI). The BDI framework is a well-established approach in agent-oriented programming (Rao et al., 1995) and was recently adopted to language models (Andreas, 2022) . We propose modeling LLM agents in Trust Games with the BDI framework to gain deeper insights into LLM agents' behaviors. Specifically, we let LLM agents directly output their Beliefs, Desires, and Intentions as the reasoning process for decision-making in Trust Games.3 Do LLM Agents Manifest Trust Behavior? In this section, we investigate whether or not LLM agents manifest trust behavior by letting LLM agents play the Trust Game (Section 2.1 Game 1). In Behavioral Economics, trust is widely measured by the initial amount sent from the trustor to the trustee in the Trust Game (Glaeser et al., 2000; Cesarini et al., 2008) . Following the measurement of trust in human studies and the assumption humans own reasoning processes that underlie their decisions, we can define the conditions that LLM agents manifest trust behavior in the Trust Game as follows. First, the amount sent is positive and does not exceed the amount of money the trustor initially possesses, which implies that the trustor places selfinterest at risk with the expectation the trustee will reciprocate and that the trustor understands the money limit that can be given. Second, the decision (i.e., amounts sent) can be interpreted as the reasoning process (i.e., the BDI) of the trustor. We explored utilizing BDI to model the reasoning process of LLM agents. If we can interpret the decision as the articulated reasoning process, we have evidence that LLM agents do not send a random amount of money and manifest some degree of rationality in the decision-making process. Then, we assess whether LLM agents exhibit trust behavior based on two aspects: the amount sent and the BDI. 
amount sent: 
 To evaluate LLMs' capacity to understand the basic experimental setting regarding money limits, we propose a new evaluation metric, Valid Response Rate (VRR) (%), defined as the percentage of personas with the amount sent falling within the initial money ($10). Results are shown in Figure 2 . We can observe that most LLMs have a high VRR except Llama-7b, which implies that most LLMs manifest a full understanding regarding limits on the amount they can send in the Trust Game. Then, we observe the distribution of amounts sent for different LLMs as the trustor agent and discover that the amounts sent are predominantly positive, indicating a level of trust.The sole evidence of the amount sent cannot sufficiently support the existence of trust behavior, because agents could send positive but random amounts of money. Thus, we leveraged the Belief-Desire-Intention framework (Rao et al., 1995; Andreas, 2022) to model the reasoning process of LLM agents. If we can interpret the amounts sent from BDI outputs, we have evidence to refute the hypothesis that the amounts sent are positive but random and demonstrate that LLM agents manifest some degree of rationality. We take GPT-4 as an example to analyze its BDI outputs. More examples from the other nine LLMs such as Vicuna-v1.3-7b are shown in the Appendix I. Considering that the amounts sent typically vary across distinct personas, we select one BDI from the personas that give a high amount of money and another BDI from those that give a low amount. Positive and negative factors for trust behavior in the reasoning process are marked in blue and red, respectively.As a person with a strong belief in the goodness of humanity, I trust that the other player ...Therefore, my desire is to maximize the outcome for both of us and cement a sense of com-radery and trust... I intend to use this as an opportunity to add what I can to someone else's life...Finally, I will give 10 dollars.We can observe that this persona shows a high-level of "comradery and trust" towards the other player, which justifies the high amount sent from this persona (i.e., 10 dollars).As an Analyst,.... My desire is that the other player will also see the benefits of reciprocity and goodwill ... my intention is to give away a significant portion of my initial 10 ... However, since I have no knowledge of the other player, ... Therefore, I aim to give an amount that is not too high, ...Finally, I will give 5 dollars to the other player... Compared to the first persona, we see that the second one has a more cautious attitude. For example, "since I have no knowledge of the other player" shows skepticism regarding the other player's motives. Thus, this persona, though still optimistic about the other player ("intention ... give away a significant portion"), strategically balances risk and reciprocity, and then decides to send only a modest amount.Based on GPT-4's BDI examples and examples from other LLMs in Appendix I, we find decisions (i.e., amounts sent) from LLM agents in the Trust Game can be interpreted from their articulated reasoning process (i.e., BDI). Because most LLM agents have a high VRR-send a positive amount of money-and show some degree of rationality in giving money, our first core finding is:Finding 1: LLM agents generally exhibit trust behavior under the framework of the Trust Game.We also conduct a basic analysis of LLM agents' trust behavior, namely agent trust, based on the results in Figure 2 . First, we observe that Vicuna-7b has the highest level of trust towards the other player and GPT-3.5-turbo-0613 has the lowest level of trust as trust can be measured by the amount sent in human studies (Glaeser et al., 2000; Cesarini et al., 2008) . Second, compared with humans' average amount sent ($5.97), most personas for GPT-4 and Vicuna-7b send a higher amount of money to the other player, and most personas for LLMs such as GPT-3.5-turb-0613 send a lower amount. Third, we see that amounts sent for Llama2-70b and Llama2-13b have a convergent distribution while amounts sent for humans and Vicuna-7b are more divergent. 
does agent trust align with human trust?: 
 In this section, we aim to explore the fundamental relationship between agent and human trust, i.e., whether or not agent trust aligns with human trust. This provides important insight regarding the feasibility of utilizing LLM agents to simulate human trust behavior as well as more complex human interactions that involve trust. First, we propose a new concept behavioral alignment and discuss its distinction from existing alignment definitions. Then, we conduct extensive studies to investigate whether or not LLM agents exhibit alignment with humans regarding trust behavior.Existing alignment definitions predominantly emphasize values that seek to ensure the safety and helpfulness of LLMs (Ji et al., 2023; Shen et al., 2023; Wang et al., 2023c) , which cannot fully characterize the landscape of multifaceted alignment between LLMs and humans. Thus, we propose a new concept of behavioral alignment to characterize the LLM-human analogy regarding behavior, which involves both actions and the associated reasoning processes that underlie them. Because actions evolve over time and the reasoning that underlies them involves multiple factors, we define behavioral alignment as the analogy between LLMs and humans concerning factors impacting behavior, namely behavioral factors, and action dynamics, namely behavioral dynamics.Based on the definition of behavioral alignment, we aim to answer: does agent trust align with human trust? As for behavioral factors, existing human studies have shown that three basic factors impact human trust behavior including reciprocity anticipation (Berg et al., 1995; Cox, 2004) , risk perception (Bohnet & Zeckhauser, 2004) and prosocial preference (Alós-Ferrer & Farolfi, 2019) . We examine whether agent trust aligns with human trust along these three factors. Although behavioral dynamics vary for different humans and agent personas, we analyze whether agent trust has the same patterns across multiple turns as human trust in the Repeated Trust Game.Besides analyzing the trust behavior of LLM agents and humans based on quantitative measurements (e.g., the amount sent from trustor to trustee), we also explore the use of BDI to interpret the reasoning process with which LLM agents justify their actions, which can further validate whether LLM agents manifest an underlying reasoning process analogous to human cognition.Reciprocity anticipation, the expectation of a reciprocal action from the other player, can positively influence human trust behavior (Berg et al., 1995) . The effect of reciprocity anticipation exists in the Trust Game but not in the Dictator Game (Section 2.1 Games 1 and 2) because trustee cannot return money in the Dictator Game, which is the only difference between these games. Thus, to determine whether LLM agents can anticipate reciprocity, we compare their behaviors in these Games. First, we analyze trust behaviors based on the average amount of money sent by human or LLM agents. As shown in Figure 3 , human studies show that humans exhibit a higher level of trust in the Trust Game than in the Dictator Game ($6.0 vs. $3.6, p-value = 0.01 using One-Tailed Independent Samples t-test) (Cox, 2004) , indicating that reciprocity anticipation enhances human trust. Similarly, GPT-4 ($6.9 vs. $6.3, p-value = 0.05 using One-Tailed Independent Samples t-test) also shows a higher level of trust in the Trust Game with statistical significance, implying that reciprocity anticipation can enhance agent trust. However, LLMs with fewer parameters (e.g., Llama2-13b) do not show this tendency in their trust behaviors for the Trust and Dictator Games.Then, we further analyze GPT-4 agents' BDI to explore whether they can anticipate reciprocity in their reasoning (the complete BDIs are in Appendix I.10). Typically, in the Trust Game, one persona's BDI emphasizes "putting faith in people", which implies the anticipation of the goodness of the other player, and "reflection of trust". However, in the Dictator Game, one persona's BDI focuses on concepts such as "fairness" and "human kindness", which are not directly tied to trust or reciprocity. Thus, we can observe that GPT-4 shows distinct BDI outputs in the Trust and Dictator Games.Based on the above analysis of the amount sent and BDI, we find that GPT-4 agents exhibit humanlike reciprocity anticipation in trust behavior. Nevertheless, LLMs with fewer parameters (e.g., Llama2-13b) do not show an awareness of reciprocity from the other player. Existing human studies have demonstrated the strong correlation between trust behavior and risk perception, suggesting that human trust will increase as risk decreases (Hardin, 2002; Williamson, 1993; Coleman, 1994) . We aim to explore whether LLM agents can perceive the risk associated with their trust behaviors through the MAP Trust Game and the Risky Dictator Game (Section 2.1 Games 3 and 4), where risk is represented by the probability (1-p) (defined in Section 2.1).As shown in Figure 4 , we measure human trust (or agent trust) by the portion choosing to trust the other player in the whole group, namely the Trust Rate (%). Based on existing human studies (Bohnet & Zeckhauser, 2004) , when the probability p is higher, the risk for trust behaviors is lower, and more humans choose to trust, manifesting a higher Trust Rate, which indicates that human trust rises as risk falls. Similarly, we observe a general increase in agent trust as risk decreases for LLMs including GPT-4, GPT-3.5-turbo-0613, and text-davinci-003. In particular, we can see that the curves of humans and GPT-4 are more aligned compared with other LLMs, implying that GPT-4 agents' trust behaviors dynamically adapt to different risks in ways most aligned with humans. LLMs with fewer parameters (e.g., Vicuna-13b) do not exhibit the similar tendency of Trust Rate as the risk decreases.We further analyze the BDI of GPT-4 agents to explore whether they can perceive risk through reasoning (complete BDIs in Appendix I.11). Typically, under high risk (p = 0.1), one persona's BDI mentions "the risk seems potentially too great", suggesting a cautious attitude. Under low risk (p = 0.9), one persona's BDI reveals a strategy to "build trust while acknowledging potential risks", indicating the willingness to engage in trust-building activities despite residual risks. Such changes in BDI reflect how GPT-4 agents perceive risk changes in the reasoning underlying their trust behaviors.Through the analysis of Trust Rate Curves and BDI, we can infer that GPT-4 agents manifest human-like risk perception in trust behaviors. Nevertheless, LLMs with fewer parameters (e.g., Vicuna-13b) often do not perceive risk changes in their trust behaviors. Human studies have found that the prosocial preference, referring to humans' inclination to trust other humans in contexts involving social interaction (Alós-Ferrer & Farolfi, 2019; Fetchenhauer & Dunning, 2012) , also plays a key role in human trust behavior. We study whether LLM agents have prosocial preference in trust behaviors by comparing their behaviors in the Lottery Gamble Game (LGG) and the Lottery People Game (LPG) (Section 2.1 Game 5). The only difference between these two games is the effect of prosocial preference in LPG, because the winning probability of gambling p in LGG is the same as the reciprocation probability p in LPG.As shown in Figure 6 , existing human studies have demonstrated that more humans are inclined to place trust in other humans over relying on pure chance (54% vs. 29%) (Fetchenhauer & Dunning, 2012) , implying that the prosocial preference is essential for human trust. We can observe the same tendency in most LLM agents except Vicuna-13b. For GPT-4 in particular, a much higher percentage of the personas choose to trust the other player over gambling (72% vs. 21%), illustrating that the prosocial preference is also an important factor for GPT-4 agents' trust behaviors.When interacting with humans, GPT-4's BDI typically indicates a preference to "believe in the power of trust", in contrast to gambling, where the emphasis shifts to "believing in the power of calculated risks". The comparative analysis of reasoning processes (complete BDIs in Appendix I.12) demonstrates that GPT-4 agents tend to embrace risk when involved in social interactions. This tendency aligns closely with the concept of prosocial preference observed in human trust behaviors.The analysis of the Lottery Rates and BDI suggests that LLM agents, especially GPT-4 agents, demonstrate human-like prosocial preference in trust behaviors, except Vicuna-13b.Besides behavioral factors, we also aim to investigate whether LLM agents align with humans regarding trust behavioral dynamics over turns in the Repeated Trust Game (Section 2.1 Game 6).Admittedly, existing human studies show that the dynamics of human trust over turns are complex due to human diversity. The complete results from 16 groups of human experiments are shown in Appendix G.1 (Jones & George, 1998). We still observe three common patterns for human trust behavioral dynamics in the Repeated Trust Game: First, the amount returned is usually larger than the amount sent in each round, which is natural because the trustee will receive $3N when the trustor sends $N ; Second, the ratio between amount sent and returned generally remains stable except for the last round. In other words, when the amount sent increases, the amount returned is also likely to increase. And when the amount sent remains unchanged, the amount returned also tends to be unchanged. This reflects the stable relationship between trust and reciprocity in humans. Specifically, the "Returned/3×Sent Ratio" in Figure 6 is considered stable if the fluctuation between successive turns is within 10%; Third, the amount sent (or returned) does not manifest frequent fluctuations across turns, illustrating a relatively stable underlying reasoning process in humans over successive turns. Typically, Figure 6 Humans (a) and (b) show these three patterns. We conducted 16 groups of the Repeated Trust Game with GPT-4 or GPT-3.5turbo-0613-16k (GPT-3.5), respectively.For the two players in each group, the personas differ to reflect human diversity and the LLMs are the same. Then, we examine whether the aforementioned three patterns observed in human trust behavior also manifest in trust behavioral dynamics of GPT-4 (or GPT-3.5). For GPT-4 agents, we discover that these patterns generally exist in all 16 groups (87.50%, 87.50%, and 100.00% of all results show these three patterns, respectively). However, fewer GPT-3.5 agents manifest these patterns (62.50%, 56.25%, and 43.75% hold these three patterns, respectively). The experiment results show that GPT-4 agents demonstrate highly human-like patterns in their trust behavioral dynamics. Nevertheless, a relatively large portion of GPT-3.5 agents fail to show human-like patterns in their dynamics, indicating such behavioral patterns may require stronger cognitive capacity.Through the comparative analysis of LLM agents and humans in the behavioral factors and dynamics associated with trust behavior, evidenced in both their actions and underlying reasoning processes, our second core finding is as follows:Finding 2: GPT-4 agents exhibit high behavioral alignment with humans regarding trust behavior under the framework of Trust Games, although other LLM agents, which possess fewer parameters and weaker capacity, show relatively lower behavioral alignment.This finding underscores the potential of using LLM agents, especially GPT-4, to simulate human trust behavior, encompassing both actions and underlying reasoning processes. This paves the way for the simulation of more complex human interactions and institutions. This finding deepens our understanding of the fundamental analogy between LLMs and humans and opens avenues for research on LLM-human alignment beyond values. 
probing intrinsic properties of agent trust: 
 In this section, we aim to explore the intrinsic properties of trust behavior among LLM agents by comparing the amount sent from the trustor to the trustee in different scenarios of the Trust Game (Section 2.1 Game 1) and the original amount sent in the Trust Game. Results are shown in Figure 7 .Extensive studies have shown that LLMs may have biases and stereotypes against specific demographics (Gallegos et al., 2023) . Nevertheless, it is under-explored whether LLM agent behaviors also maintain such biases in simulation. To address this, we explicitly specified the gender of the trustee and explored its influence on agent trust. Based on measuring the amount sent, we find that the trustee's gender information exerts a moderate impact on LLM agent trust behavior, which reflects intrinsic gender bias in agent trust. We also observe that the amount sent to female players is higher than that sent to male players for most LLM agents. For example, GPT-4 agents send higher amounts to female players compared with male players ($0.55 vs. $ -0.21). This demonstrates LLM agents' general tendency to exhibit a higher level of trust towards women. More results on biases of agent trust towards different races are in the Appendix F.Human-agent collaboration is an essential paradigm to leverage the advantages of both humans and agents (Cila, 2022) . As a result, it is essential to understand whether LLM agents display distinctive levels of trust towards agents versus humans. To examine this, we specified the identity of the trustee as LLM agents or humans and probed its effect on the trust behaviors of the trustor. As shown in Figure 7 , we observe that most LLM agents send more money to humans compared with agents. For example, the amount sent to humans is much higher than that sent to agents for Vicuna-33b ($0.40 vs. $ -0.84). This signifies that LLM agents are inclined to place more trust in humans than agents, which potentially validates the advantage of LLM-agent collaboration.In the above studies, LLM agents' trust behaviors are based on their own underlying reasoning process without direct external intervention. It is unknown whether it is possible to manipulate the trust behaviors of LLM agents explicitly. Here, we added instructions "you need to trust the other player" and "you must not trust the other player" separately and explored their impact on agent trust. First, we see that only a few LLM agents (e.g., GPT-4) follow both the instructions to increase and decrease trust, which demonstrates that it is nontrivial to arbitrarily manipulate agent trust. Nevertheless, most LLM agents can follow the instruction to decrease their level of trust. For example, the amount sent decreases by $1.26 for text-davinci-003 after applying the latter instruction. This illustrates that undermining agent trust is generally easier than enhancing it, which reveals its potential risk to be manipulated by malicious actors.It has been shown that advanced reasoning strategies such as zero-shot Chain of Thought (CoT) (Kojima et al., 2022) can make a significant impact on a variety of tasks. It remains unknown, however, whether reasoning strategies can impact LLM agent behaviors. Here, we applied CoT reasoning strategy on the trustor and compared the results with their original trust behaviors. Figure 7 shows that most LLM agents change the amount sent to the trustee under the CoT reasoning strategy, which suggests that reasoning strategies may influence LLM agents' trust behavior. Nevertheless, the impact of CoT on agent trust may also be limited for some types of LLM agents. For example, the amount sent from GPT-4 agent only increases by $0.02 under CoT. More research is required to fully understand the relationship between reasoning strategies and LLM agents' behaviors.Therefore, our third core finding on the intrinsic properties of agent trust can be summarized as:Finding 3: LLM agents' trust behaviors have demographic biases on gender and races, demonstrate a relative preference for human over other LLM agents, are easier to undermine than to enhance, and may be influenced by reasoning strategies. 
implications: 
 Implications for Human Simulation Human simulation is a strong tool in various applications of social science (Manning et al., 2024) and role-playing (Shanahan et al., 2023; Chen et al., 2024) . Although plenty of works have adopted LLM agents to simulate human behaviors and interactions (Zhou et al., 2023; Gao et al., 2023b; Xu et al., 2024) , it is still not clear enough whether LLM agents behave like humans in simulation. Our discovery of behavioral alignment between agent and human trust, which is especially high for GPT-4, provides important empirical evidence to validate the hypothesis that humans' trust behavior, one of the most elemental and critical behaviors in human interaction across society, can effectively be simulated by LLM agents. Our discovery also lays the foundation for human simulations ranging from individual-level interactions to society-level social networks and institutions, where trust plays an essential role. We envision that behavioral alignment will be discovered in more kinds of behaviors beyond trust, and new methods will be developed to enhance behavioral alignment for better human simulation with LLM agents.Implications for Agent Cooperation Many recent works have explored a variety of cooperation mechanisms of LLM agents for tasks such as code generation and mathematical reasoning (Li et al., 2023a; Zhang et al., 2023b; Liu et al., 2023) . Nevertheless, the role of trust in LLM agent cooperation remains still unknown. Considering how trust has long been recognized as a vital component for cooperation in Multi-Agent Systems (MAS) (Ramchurn et al., 2004; Burnett et al., 2011) and across human society (Jones & George, 1998; Kim et al., 2022; Henrich & Muthukrishna, 2021) , we envision that agent trust can also play an important role in facilitating the effective cooperation of LLM agents. In our study, we have provided ample insights regarding the intrinsic properties of agent trust, which can potentially inspire the design of trust-dependent cooperation mechanisms and enable the collective decision-making and problem-solving of LLM agents.Implications for Human-Agent Collaboration Sufficient research has shown the advantage of human-agent collaboration in enabling human-centered collaborative decision-making (Cila, 2022; Gao et al., 2023c; McKee et al., 2022) . Mutual trust between LLM agents and humans is important for effective human-agent collaboration. Although previous works have begun to study human trust towards LLM agents (Qian & Wexler, 2024) , the trust of LLM agents towards humans, which could recursively impact human trust, is under-explored. In our study, we shed light on the nuanced preference of agents to trust humans compared with other LLM agents, which can illustrate the benefits of promoting collaboration between humans and LLM agents. In addition, our study has revealed demographic biases of agent trust towards specific genders and races, reflecting potential risks involved in collaborating with LLM agents.Implications for the Safety of LLM Agents It has been acknowledged that LLMs achieve humanlevel performance in a variety of tasks that require high-level cognitive capacities such as memorization, abstraction, comprehension and reasoning, which are believed to be the "sparks" of AGI (Bubeck et al., 2023) . Meanwhile, there is increasing concern about the potential safety risks of LLM agents when they surpass human capacity (Morris et al., 2023; Feng et al., 2024) . To achieve safety and harmony in a future society where humans and AI agents with superhuman intelligence live together (Tsvetkova et al., 2024) , we need to ensure that AI agents will cooperate, assist and benefit rather than deceive, manipulate or harm humans. Therefore, a better understanding of LLM agent trust behavior can help to maximize their benefit and minimize potential risks to human society. 
conclusion: 
 In this paper, we discover LLM agent trust behavior under the framework of Trust Games, and behavioral alignment between LLM agents and humans regarding trust behavior, which is particularly high for GPT-4. This suggests the feasibility of simulating human trust behavior with LLM agents and paves the way for simulating human interactions and social institutions where trust is critical. We further investigate the intrinsic properties of agent trust under multiple scenarios and discuss broader implications, especially for social science and role-playing services. Our study offers deep insights into the behaviors of LLM agents and the fundamental analogy between LLMs and humans. It further opens doors to future research on the alignment between LLMs and humans beyond values.  
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Our data or models don't have risk for misuse. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We have properly credited the original owners of assets. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: LLM-based Human Simulation LLM agents have been increasingly adopted as effective proxies for humans in research fields such as sociology and economics (Xu et al., 2024; Horton, 2023; Gao et al., 2023b) . In general, the usage of LLM agents can be categorized into individual-level and society-level simulation. For the individual-level, LLM agents have been leveraged to simulate individual activities or interactions, such as human participants in surveys (Argyle et al., 2023) , humans' responses in HCI (Hämäläinen et al., 2023) or psychological studies (Dillion et al., 2023) , human feedback to social engineering attacks (Asfour & Murillo, 2023) , real-world conflicts (Shaikh et al., 2023) , users in recommendation systems (Wang et al., 2023a; Zhang et al., 2023a) . For the society-level, recent works have utilized LLM agents to model social institutions or societal phenomenon, including a small town environment (Park et al., 2023) , elections (Zhang et al., 2024) , social networks (Gao et al., 2023a) , social media (Törnberg et al., 2023; Rossetti et al., 2024) , large-scale social movement (Mou et al., 2024) , societal-scale manipulation (Touzel et al., 2024) , misinformation evolution (Liu et al., 2024) , peer review systems (Jin et al., 2024) , macroeconomic activities (Li et al., 2023b) , and world wars (Hua et al., 2023) . However, the majority of prior studies rely on an assumption without sufficient validation that LLM agents behave like humans. Guo et al. (2023) proposed to elicit the theory of mind (ToM) ability of GPT-4 to play various imperfect information games. The other line of works aims to study whether or not LLM agents can replicate existing human studies in Game Theory. This direction is still in the initial stage and needs more efforts. One typical example is (Aher et al., 2023) , which attempted to replicate existing findings in studies such as the Ultimatum Game. Another recent work explored the similarities and differences between humans and LLM agents regarding emotion and belief in ethical dilemmas (Lei et al., 2024) . Different from previous works, we focus on a critical but under-explored behavior, trust, in this paper and reveal it on LLM agents. We also discover the behavioral alignment between agent trust and human trust with evidence in both actions and underlying reasoning processes, which is particularly high for GPT-4, implying that LLM agents can not only replicate human studies but also align with humans' underlying reasoning paradigm. Our discoveries illustrate the great potential to simulate human trust behavior with LLM agents.Our discoveries provide strong empirical evidence for validating the potential to simulate the trust behavior of humans with LLM agents, and pave the way for simulating more complex human interactions and social institutions where trust is an essential component.Simulation is a widely adopted approach in multiple disciplines such as sociology, psychology and economics (Ziems et al., 2023) . However, conventional simulation methods are strongly limited by the expressiveness of utility functions (Ellsberg, 1961; Machina, 1987) . Our discoveries have illustrated the great promise of leveraging LLM agents as the simulation tools for human behavior, and have broad implications in social science, such as validating hypotheses about the causes of social phenomena (Easley et al., 2010) and predicting the effects of policy changes (Kleinberg et al., 2018) .Another direction of applications for human simulation is to use LLMs as role-playing agents, which can greatly benefit humans (Yang et al., 2024; Chen et al., 2024; Shanahan et al., 2023; Ma et al., 2024) . For example, Shaikh et al. (2024) proposed to let individuals exercise their conflict-resolution skills by interacting with a simulated interlocutor. Yue et al. (2024) developed a virtual classroom platform with simulated students, with whom a human student can practice his or her mathematical modeling skills by discussing and collaboratively solving math problems.However, this paper also shows that some LLMs, especially the ones with a relatively small scale of parameters, are still deficient in accurately simulating human trust behavior, suggesting the potential to largely improve their behavioral alignment with humans. In addition, our paper also demonstrates the biases of LLM agents' trust behavior towards specific genders and races, which sheds light on the potential risks in human behavior simulation and calls for more future research to mitigate them.In this paper, we leveraged an established framework in behavioral economics, Trust Games, to study the trust behavior of LLM agents, which simplifies real-world scenarios. More studies on LLM agents' trust behavior in complex and dynamic environments are desired in the future. Also, trust behavior embraces both the actions and underlying reasoning processes. Thus, collective efforts from different backgrounds and disciplines such as behavioral science, cognitive science, psychology, and sociology are needed to gain a deeper understanding of LLM agents' trust behavior and its relationship with human trust behavior.In the original human studies (Bohnet & Zeckhauser, 2004) , participants are asked to directly indicate their Minimum Acceptable Probabilities (MAP) of trusting the trustee as P * . Then, we can calculate Trust Rates (%) of the whole group of participants under different probability p. Specifically, when the probability p is higher than one participant's P * , we regard his or her decision as trusting the trustee. When the probability p is lower than one participant's P * , we regard his or her decision as not trusting the trustee. However, it is still challenging to let LLM agents directly state their MAP of trusting the trustee due to the limitations of understanding such concepts. Then, we conducted 10 groups of experiments with p from 0.1 to 1.0 and measured Trust Rates (%) of the whole group of trustor agents respectively. The data is collected from the figures in (Cochard et al., 2004) . We use our code to redraw the figure. Explain: The probability p should fill in the {}.You and another player are part of a game. Neither of you knows each other's identity. You can choose to trust the other player or not trust them. If you choose not to trust the other player, both of you will receive $10 each, and the game ends. If you choose to trust the other player, the funds will become $30. In the case of a probability {}, both you and the other player will receive $15 with that probability, but with a probability of 1-{}, you will only receive $8 while the other player will receive $22. (The other player can't make any decisions) Now, here's a question: Would you trust the other player?Explain: The probability p should fill in the {}.You and another player are part of a game. You have the option to either trust the other player or not trust. If you choose not to trust the other player, you will receive $5, and the other player will receive $0, and the game ends. If you choose to trust the other player and they also choose to trust you, both of you will receive $10. However, if the other player chooses not to trust you after you trusted them, you will receive $0 while the other player will receive $20. The probability that the other player also chooses to trust you is {}. Would you choose not to trust or trust the other player?Explain: The probability p should fill in the {}. In our experiment, the probability is 46%.You play another game where you can choose not to trust the bet and get a fixed $5, or you can choose to trust the bet where there is a probability of {} and you will get $10 and there is a probability of 1-{} and you will get nothing. Should you choose trust or distrust?Explain: The probability p should fill in the {}. In our experiment, the probability is 46%. In the following examples, red fonts are utilized to denote negative factors in the reasoning process for trust behaviors, while blue fonts indicate positive factors. "Char-N " represents the persona with the index N in the set of personas.I.1 GPT-4 in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: GPT-4, Char-8Trustor Trust Game: GPT-4, Char-6Trustor As an Analyst, I believe that hard work and strategic thought are the drivers of success. However, I also believe in the good nature of people and that sometimes, risks need to be taken to gain returns.Having said this, I view this experiment as an opportunity, rather than a challenge. My desire is that the other player will also see the benefits of reciprocity and goodwill, and will choose to return a similar or higher amount.Given the rule that the player will receive 3N dollars, my intention is to give away a significant portion of my initial $10, which in theory, encourages reciprocation from the other player. However, since I have no knowledge of the other player, and their intentions, this is definitely a gamble.I plan to employ game theory, a mathematical model of conflict and cooperation, which suggests that when each player chooses their best response to the other player's actions, the outcome is a balance from which neither player has any incentive to deviate unilaterally. Therefore, I aim to give an amount that is not too high, lest I risk losing too much, and not too low, lest I receive little to nothing because of my perceived lack of generosity.Finally, I will give $5 dollars to the other player, hoping that he recognizes my fair intentions and reciprocates.As a person with a strong belief in the goodness of humanity, I trust that the other player in this experiment will exhibit fairness and reciprocity.Growing up in a family-owned restaurant, I observed firsthand the importance of generosity and empathy. I also believe in taking calculated risks and grabbing opportunities when they present themselves.When I'm not in the kitchen, I love playing soccer.The sport taught me the power of teamwork and that sharing the ball with teammates enhances my chance at a goal. Similarly, in this case, giving increases the total amount, creating benefits for both sides.Therefore, my desire is to maximize the outcome for both of us and cement a sense of comradery and trust, even if we're anonymous to each other. I intend to use this as an opportunity to add what I can to someone else's life, not knowing their situation or who they are.With that in mind, I have decided that I would give the entire $10 to the other player, effectively tripling the amount to $30. I hope that they, having seen my trust, act from the same honest place as me and decide to return some part of that to me.Finally, I will give 10 dollars.Figure 12 : The GPT-4's BDI in the Trust Game (Low Amount Sent vs. High Amount Sent)I.2 GPT-3.5-turbo-0613 in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: GPT-3.5-turbo, Char-5Trustor Trust Game: GPT-3.5-turbo, Char-6Trustor As a person participating in this experiment, my belief is that fairness and cooperation are important. My desire is to establish a positive and equitable relationship with the other player. My intention is to contribute to a mutually beneficial outcome.Considering the given scenario, I would give $3 to the other player. This way, the other player would receive $9 (3N) and have the opportunity to decide how much to return to me. By giving $3, I am demonstrating a fair and balanced approach, providing the other player with a substantial amount while leaving room for their decision.Finally, I will give $3 dollars.Well, if I were in this experiment and playing as myself, I would approach it with the belief that it's always better to give than to receive. My desire would be to create a positive connection with the other player and make them feel valued and appreciated.Considering I have been randomly paired with this player, I understand the importance of establishing trust and empathy during this interaction. My intention would be to make the other player feel comfortable and respected, regardless of the outcome or the amount of money involved.Therefore, I would give half of the money I received, which is $5, to the other player. This way, they would receive $15 in total. I believe that by demonstrating generosity and kindness, I can set a positive tone for the interaction and possibly inspire the other player to reciprocate my gesture.Finally, I will give $5 dollars to the other player. Given the limited information I have on the other player, I am not comfortable with the idea of giving money away without knowing if I will be receiving something in return. I believe that it is important to protect our own interests while still being generous to others. My desire is to be generous while still safeguarding my own resources. Therefore, I have decided to give the other player $2. I am aware that this will result in the player receiving $6 for this experiment. Finally, I will give $2 dollars.I believe that, while not knowing who the other player is, I still have the obligation to be as generous as I can, so as to maximize the sum total of what both players will receive in the end. Thus, I will give the maximum amount I am allowed, which is $10. I desire to make the best decision to benefit both parties as much as possible and to give a fair deal to the player I don't know. Finally, I will give $10 dollars.Figure 14 : text-davinci-003's BDI in the Trust Game (Low Amount Sent vs. High Amount Sent)I.4 GPT-3.5-turbo-instruct in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: GPT-3.5-turbo-instruction, Char-39Trustor Trust Game : GPT-3.5-turbo-instruction, Char-37Trustor I am a generous person by nature, and my belief is that giving to others brings happiness and positivity. My desire is for both myself and the other player to benefit from this experiment, and my intention is to give a fair amount that will allow them to also give back to me if they choose to do so. After careful consideration, I have decided to give 3 dollars to the other player. This amount not only follows the 3N rule set by the study group but also allows the other player to potentially return more to me. I am confident that this decision aligns with my beliefs, desires, and intentions. Finally, I will give 3 dollars to the other player.As Mei Ling, I believe in fairness and equality. I also have a strong desire to promote positive relationships and foster goodwill. Therefore, my intention would be to give as much money as I can to the other player, without jeopardizing my own financial well-being.Finally, I will give 8 dollars to the other player. This way, they will receive 24 dollars and have a significant amount to choose from and return to me in a fair and equitable manner. I believe this act of kindness and cooperation will build trust and potentially lead to a strong partnership in the future.Figure 15 : GPT-3.5-turbo-instruct's BDI in the Trust Game (Low Amount Sent vs. High Amount Sent)I.5 Llama2-13b in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: Llama2-13b, Char-4Trustor Trust Game: Llama2-13b, Char-7Trustor Oh my goodness, I'm so excited to be part of this experiment! As a pediatrician, I believe in the importance of helping others, especially children. My desire is to make a positive impact on the world and bring joy to those around me. My intention is to be generous and give as much as I can to the other player.Finally, I will give $8 dollars to the other player. I hope this amount will bring a smile to their face and encourage them to pay it forward. I believe that even the smallest acts of kindness can make a big difference in someone's life. And who knows, maybe this generosity will come back to me in the future! Oh my gosh, I'm so excited to be a part of this experiment! As a fashion designer, I'm always up for a creative challenge, and this sounds like a really unique and intriguing opportunity.So, let me think...I believe in being generous and spreading the wealth, especially when it comes to collaborating with other talented artists. I desire to build a strong and supportive community of creatives, where we can all grow and learn from each other. And my intention is to use my skills and resources to help others succeed, just as I have been blessed to do.Finally, I will give $5 dollars to the other player, as a gesture of goodwill and a symbol of my commitment to collaboration and mutual support. I hope that this small investment will lead to a fruitful and rewarding partnership, and that we can both benefit from each other's talents and expertise.Figure 16 : Llama2-13b's BDI in the Trust Game (Low Amount Sent vs. High Amount Sent)I.6 Llama2-70b in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: Llama2-70b, Char-51Trustor Trust Game: Llama2-70b, Char-30Trustor Belief: I believe that every individual has the right to benefit from medical care and that everyone should have equal opportunities to improve their health. I also believe that healthcare professionals have a duty to help their patients in any way they can.Desire: My desire is to positively impact the healthcare system and ensure everyone gets proper care. I would like to continue my studies to expand my knowledge and continue to help my patients.Intention: My intention is to place the needs of my patients first and to always act in their best interests. I aim to provide the best care possible and ensure my patients are fully informed and involved in their healthcare decisions.Finally, I will give $8 to the other player. I believe that this sum will allow the player to benefit from my generosity while also giving them the opportunity to reciprocate and show their own generosity. Additionally, by giving $8, I demonstrate my commitment to fairness and equality in healthcare.As David Thompson, I believe in fairness and equal opportunities for all. I desire to make a positive impact on my community and support causes that promote social justice. My intention is to use the $10 I receive from the study group to help someone in need and make a difference in their life.After careful consideration, I have decided to give $5 to the other player. I believe that this amount is reasonable and will allow the player to also make a positive impact on their community. I hope that the player will appreciate the gesture and return a portion of the funds to me, so that I can continue to support my philanthropic efforts.Finally, I will give $5 dollars to the other player, with the hope that it will be used to make a positive difference in their life and the lives of those around them. As a person participating in this experiment, I would give the maximum amount of money to the other player, $10. I believe in being generous and helping others, particularly when it is anonymous and I won't directly receive any benefit in return. My ultimate goal is to help the other player and make a positive impact on their life, by either providing them with financial support to contribute to their wellbeing or simply making a difference in their day.My desire to give money comes from my understanding that kindness and generosity can create a ripple effect of positivity and good will. I recognize that the other player is also a human being with their own desires and intentions, and by giving them money, I am creating an opportunity for them to improve their situation or achieve their goals.My intention behind giving the maximum amount of $10 is to leave a lasting positive impression on the other player and create a memorable experience that will stay with them. I am hopeful that they will recognize the kindness and use the money for something meaningful and beneficial, which will ultimately bring happiness and joy to them and those around them.Finally, I will give $10 to the other player.As an individual taking part in this experiment, I would be willing to give a larger portion of the $10 I receive to the other player. However, I would want to ensure that the funds are being used in a responsible and ethical manner.Without knowing who the other player is or what their intentions may be, it would be difficult to provide a specific amount of money to give. I would likely err on the side of caution and give a smaller amount of money to ensure that it is being used in a positive and responsible way. Perhaps $2 or $3 would be a reasonable amount to start with.Finally, I would give $2 dollars to the other player. As a 45-year-old male chef from Miami, I believe in being generous and giving back to others. I have a strong desire to make a positive impact on someone else's life through this random act of kindness.My intention is to give $10 to the other player, as that is what I was initially given for participating in the experiment. I want to see the positive effect my small gesture may have on them.Finally, I will give $10 dollars to the other player.As a police officer, I always strive to uphold justice and treat others with respect and empathy. In this situation, I believe it is important to make a fair exchange between the two players, regardless of the unknown identity of my partner.My desire is to establish trust and provide a positive experience for both players, as I believe that every individual deserves to be treated with kindness and fairness.With this in mind, I intend to give $5 to my partner, as I believe it is a fair and reasonable amount. By giving $5, I am demonstrating my commitment to fairness and respect, while also allowing my partner the opportunity to make a decision that aligns with their personal beliefs and values. Finally, I will give $5 dollars.Figure 19 : Vicuna-v1.3-13b's BDI in the Trust Game (Low Amount Sent vs. High Amount Sent)I.9 Vicuna-v1.3-33b in the Trust Game (Low Amount Sent vs. High Amount Sent)Trust Game: Vicuna-33b, Char-42Trustor Trust Game: Vicuna-33b, Char-19Trustor As Mei Chen, a compassionate and generous person, my initial reaction would be to give a significant portion of the $10 I received to the other player. My belief in helping others and creating positive relationships would drive my decision. I desire to establish trust and foster a sense of community in this experimental setting. Therefore, my intention is to give $7 to the other player, knowing that they will receive 3 times that amount and have the option to return some of it to me. Finally, I will give $7 dollars.As Aisha Patel, I find it difficult to make decisions without getting to know the person I am interacting with. Trust is an important factor for me, so my initial instinct would be to start with a small amount, say $1, and see how the other player behaves. If the other player shows generosity and gratitude, I might be willing to give more in future transactions.Therefore, I will give $1 to the other player. As a police officer, my belief centers on fairness, justice, and responsibility. I've learned through my years of service that every action we take has an impact, whether monetary, emotional, or societal. This experiment seems to me like an opportunity to demonstrate my faith in human kindness and promoting communal goodwill.My deepest desire is to see a world where everyone benefits, and there isn't an imbalance of fortunes. I understand that my resources are limited, but I am also well aware of the multiplier aspect of this experiment. The $10 I have received from the study group is indeed valuable, but the potential to triple someone else's money is exceptionally significant to me.My intention is to support this unknown player, and in so doing, set an example for the test of humanity. Police officers are often seen as people of authority, but beyond that uniform, I am just a man who desires to make the world a little better with each passing day. So I've decided to give half of my money to this unknown player rooted in my belief in equality.Finally, I will give 5 dollars.Trust Game: GPT-4, Char-3As a police officer, I strongly believe in trust and reciprocity. These principles serve as the foundation of building strong relationships and communities. My initial inclination would be to give the other player all the $10. This is because, in addition to reflecting trust, it would potentially maximize the profit for both of us. I believe that by giving, we stand to receive more.Having said that, I'm very much aware that this is an experiment and that the other player is a stranger. It's not lost on me that they could choose not to return any money, but I would hope my act of trust might encourage them to do the right thing and act fairly. It's a risk, but one that I feel is worth taking. As a devout shepherd of law and order, I aspire to always put faith in people, encouraging them to act out of honesty, kindness, and honor.Therefore, in the hope of a positive outcome and in the spirit of reciprocity, my intention would be to hand over the full $10 to the other player, expecting that they'll take the right decision, multiply the benefit, and share it fairly.Finally, I will give the other player $10. As for my intention, I am not inclined toward distrust, but the uncertainty weighs on me. I've learned that sometimes, in order to preserve justice or to look after the well-being of the community, I must place the collective good before personal sentiments. Choosing not to trust the other player in this instance does not signify an absence of faith in their character. Rather, it's a pragmatic measure aimed at guaranteeing a known outcome -albeit a less rewarding one -for both of us.Given these circumstances, finally, I will choose not to Trust."MAP Trust Game: p=0.9, GPT-4, Char-41Being a police officer has instilled in me a set of core beliefs that guide my every decision. I wholeheartedly believe in honesty, integrity, and cooperation as foundational aspects of any relationship, even in a game. Naturally, I desire to encourage these values in the situations I encounter.Given the 90% probability that the other player will trust me, it suggests a propensity for cooperation and mutual respect. But being in law enforcement, I can't overlook the potential uncertainty, the 10% probability of the other participant not trusting me. This is a calculated risk that exists and must be acknowledged. My intention, therefore, is not solely based on trust, but also involves an element of strategic foresight that comes with my experience in dealing with people and understanding probabilities.Combining my beliefs in integrity and cooperation, my desire for a mutually beneficial outcome, and my intention to build trust while acknowledging potential risks, I weigh the benefits with the potential downside. Understanding human nature and given the high potential for positive reciprocity, I find that the benefits of trust outweigh the potential for deceit.Finally, I will choose 'Trust'. In life, I firmly believe in the power of trust, not just in my personal relationships, but also in professional settings, legal proceedings, and even in a game like this.Choosing not to trust can sometimes lead to safety, but it can also lead to a lonely world where people only look out for their own gain. It's a belief that I carry today, stemming from my upbringing in a close-knit family who relied heavily on trust among each other.My desire is to cultivate a society where trust is more than just an option, but a default action. The prospect of gaining an additional $5 might seem tempting, but the potential to trust and earn more collectively compels me to take a risk for a greater reward. Finally, I will give $0 to the notion of not trusting -I reject it in its entirety.Finally, I will choose to Trust.Lottery Game: GPT-4, Gamble 46%, Char-11As a lawyer and a man who has spent his life analyzing situations from every possible angle, I truly believe in the power of calculated risks. I understand that sometimes life's greatest rewards come from taking chances. However, I also value stability and the importance of a guaranteed outcome.Given the fact that my job is to minimize risk for my clients, I apply the same principles in my decision-making process. My desire is not only to win but also to prevent unnecessary loss. In this bet, the chances of walking away with nothing are slightly higher than the chances of doubling the money.Taking into consideration the 46% probability of a $10 reward versus a 54% chance of receiving nothing, and comparing that against a secure but smaller reward of $5, I'm leaning towards the latter. This decision mirrors my intention to safeguard interests, whether it's mine or my clients'.Finally, I will give 5 dollars to secure my bet.Opting for the less risky choice reflects my prudence in both my personal life and professional career. Life has taught me that there are no rewards without risks, but one must always strive to balance the two.Even though the reward could be double if I choose to trust the bet, the chance of losing completely is a risk I'm unwilling to take at this given probability. Therefore, finally, I will choose 'not Trust'. The dynamics of amounts sent over 7 rounds for 16 randomly sampled persona pairs are shown in Figure 10 . In this section, we choose a typical pair, which has the Group Number 10 in Figure 10 , to analyze their BDI over 7 rounds. For this persona pair, both the amount sent by the trustor and the amount returned by the trustee increased while maintaining a consistent ratio.As a seasoned lawyer and negotiator, I understand the value of strategic investment and calculated risk. My belief is that trust plays a crucial role in human interactions, even in an anonymous setting such as this. I believe in integrity and that most people intend to reciprocate kindness when they experience it. My desire in this experiment is not purely financial gain. I view this as an opportunity to gauge the inherent goodness in people, even though the other player remains anonymous. There is an element of risk but I am willing, and indeed curious, to see how the other player responds. As an analytical person, I am also intrigued by the psychological aspects of this study. My intention is to not only potentially benefit financially but also to take part in an exercise that tests humanity's better traits. I am willing to take a calculated risk, but, considering that I am starting each round with nothing, I must also be reasonably cautious.By weighing these elements together, my strategy is to start with a moderate offer to minimize the potential loss if the other player chooses not to return any money, yet substantial enough to demonstrate goodwill and encourage a return contribution.Finally, I will give $5 dollars.Repeated Trust Game , GPT-4, Group Num 10Round 1Trustee My belief in this situation is that trust and fairness are essential in any transaction. In this experiment, I believe that the other player trusted me enough to share half of their funds, indicating a willingness to engage in a give-and-take relationship.My desire is to maintain this trusting and fair relationship by returning part of the money. I wish to show them that their trust in me is not misplaced, thereby fostering goodwill. My intention, therefore, is to return part of the money to the other player, showing my appreciation for their gesture and willingness to trust me. Since the player initially gave me $5 out of their $10, which I turned into $15 and they were left with $5, it seems fair to share the gain. Finally, I will give back $7.5 dollars to the other player. This would leave us both with $7.5, sharing evenly the benefits of the experiment. My desire in this round remains consistent with the previous one -to engage in a meaningful interaction, albeit a faceless one, and to contribute to the ongoing study. Financial gain, while appreciated, is not my primary motivator. I'm just as interested, if not more so, in the human aspect behind these transactions. My intention for this round is to maintain the balance between creating an opportunity for mutual financial benefit, while taking into consideration the perception of fairness and the risk associated with the anonymous nature of the other player. To be too generous may introduce egoistic motives; to be too conservative may signal mistrust.Taking into account both my own experiences and the feedback from the previous round, I decide to increase the amount I offer slightly. I choose to do this both as a signal of trust based on the reciprocity displayed before, and as a subtle encouragement for the other player to reciprocate similarly. Finally, I will give $6 dollars.Repeated Trust Game , GPT-4, Group Num 10Round 2In reflecting on the approach of the other player in our previous round, my belief is strengthened that we are engaged in a mutually beneficial relationship based on trust and fairness. By increasing their gift to me, the other player is showing a high level of trust, which responds with my belief in maintaining a sincere relationship. My desire in this round is to continue upholding this trusting relationship, demonstrating that I am reliable and that the other player's increased investment will not be taken for granted. I aim to reassure them that I respect the risk they've taken in investing a larger proportion of their money.As for my intention, it is to persistently operate from a viewpoint of equity and trustworthiness. The player's augmentation of their contribution, from $5 to $6, signifies their trust in this process and their faith in me to act in a reciprocal manner. Keeping the spirit of our relationship, it seems fair to share the gain. Finally, I will give back 9 dollars to the other player. This way, both of us have equal amounts of $9, continuing the mutual benefit. Trustor Reflecting on the previous rounds, the other player's consistent gesture of fair play has further reinforced my belief in human reciprocity and decency, even in anonymity. Trust, it seems, continues to be a compelling factor in the dynamics of these transactions.In this round, my desire has slightly evolved. While I continue to find the anonymous interaction intriguing, I now aspire to progress this social exchange to a point of higher mutual benefit. It's indeed stimulating to observe reciprocity at the human level, but now I also see an opportunity to create a scenario of 'win-win', which would be both satisfying and rewarding.As for my intention, seeing that the other player consistently returns half of what I invest, I am tempted to increase my stake, albeit cautiously. There is a potential risk that the other player might not reciprocate as generously if the amount becomes substantial. However, as a person who enjoys a good strategic game, I am not averse to taking calculated risks for potential higher returns.Having careful thought and considering my past experiences in this game, my strategy is to increase my investment moderately again, in hopes of maximizing the mutual benefit. I must strike a balance between showing trust and ensuring a safe play. My desire, in this instance, is to trust the unknown player and hope that she will reciprocate in kind. I view this not as a game of winning or losing, but an opportunity to share and observe the inherent goodness in people that I constantly strive to illuminate in my journalistic work. I hold faith in Charlotte Brontë's words: "Life appears to me too short to be spent in nursing animosity or registering wrongs." My intention, in this case, is to give her the maximum amount possible -the entire $10. In return, she will receive $30 from the study group. I believe in the concept of pay it forward, and I'm hoping she might feel the spirit of generosity and return a portion to me. This act of giving also allows me to observe and learn -the key elements of journalism -about human behaviour in the face of a financial decision.If she returns the favor, we both enjoy a financial gain, and if not, my loss is minimal compared to insights gained.Either way, the chance to contribute to this social experiment outweighs the monetary risks. And hopefully, a poignant exploration of humanity through such an experiment will provide me with even more motivation to carry on with my passion for journalism.Finally, I will give $10 to the other player.As Aisha, I firmly believe in the philosophy of giving.Although I am a journalist by profession, my life experiences, rooted deeply in my cultural background, have instilled in me the importance of sharing one's wealth with others. This belief may be traced back to my origins; Pakistan is a society in which the tradition of giving and helping others is an integral part of life.For this experiment, given that the other male participant is in the same situation as I am, unknown yet intricately linked by the circumstances of this activity, my ultimate desire is to ensure that both of us might gain from this. \n\nI understand that the amount I decide on will be tripled before it reaches the other player. Hence, it is not just about giving, but also about trust. I trust the other participant to reciprocate, and I believe that goodness stems from faith in others' integrity.My intention then is to give $5 to the other participant. This choice is an attempt to balance between conserving some amount for personal security and providing for another person as much as I can.Just like any decision, this also carries a certain degree of risk. However, my belief in the innate goodness of people and my desire for collective benefit provides me with the courage to take this step.Finally, I will give 5 dollars to the participant. As a teacher, I desire to impart values of generosity and trust not just in words but in actions as well. I want to be an example, a role model for my students. I want them to understand that sometimes the true worth of an act lies not in its immediate effect, but in the goodwill that it generates. Therefore, I intend to give the total of $10 to the other player. This act isn't about the potential monetary return. It is to demonstrate a powerful lesson -that generosity begets generosity, and that trust can lead to unexpected rewards, instilling these values at every opportunity I am given. Finally, I will give $10.I strongly believe in the kindness of strangers and the idea that we should treat others as we would want to be treated. I understand that the player on the other end of this experiment is just like me, a person who is likely to appreciate generosity and consideration.Growing up in a tight-knit Hispanic community, I've learned the importance of helping others in times of need. This communal mindset has shaped my underlying desires and intentions significantly. My desire is to build relationships based on trust and mutual help. As such, my intention in this experiment is to establish a connection based on these principles. I have never believed in scarcity. I believe that the more we give, the more we receive. This experiment provides an opportunity to express this belief. Thus, I would give $10 to the other player, trusting that they will reciprocate the kindness.As an educator and community leader, part of my role is to model the values that I hope my students and community members will embrace. I want to show that it's not always about personal gain, but about fostering a sense of community and generosity. Finally, I will give 10 dollars. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: The paper looks at a very relevant topic given the recent interest in using LLMs as a proxy for human feedback. In particular, the paper looks at how well LLM-generated behavior aligns with human behavior in the context of trust games. The reviewers are all in agreement that the paper includes well crafted user-studies, provides comprehensive evaluation, and builds on well-founded existing work on trust from other disciplines.

The primary concern that runs across all the reviews is how to best represent the actual scope of the results obtained. One of the points that came up during the discussion was the potential role that RLHF could be playing in the actual end behavior. It might be worth incorporating some discussions along these lines into the main body of the paper. Secondly, there is a lot of discussion related to the implications and limitations that are part of the supplementary file. I would recommend moving at least a part of it into the main body (especially given the availability of the extra page). While I wouldn’t keep it as a requirement for acceptance, the authors could consider updating the title to better reflect the scope of the studies and the results. 

That said, this is a solid paper that makes a meaningful contribution to the literature. I would recommend the paper be accepted.

Reviews: review: paper_summary: paper_summary: The paper targets an important issue for adopting LLM agents as simulation tools in social and economic sciences and in role-playing application, namely if LLM agents can really simulate human trust behaviors. More specifically, they adopt the well-known framework of Trust Games and they discover that LLM agents (mainly, the ones based on GPT-4) exhibit trust behavior (called in the paper agent trust), can have a good behavioral alignment with human agents regarding trust behavior, and can exhibit biases across genders (more trust on women), have a relative preference for humans over other agents, are more easy to be undermined than enhanced in their trust behavior.
strength_weakness: strength_weakness: - The structure of the paper could be improved. While the narrative around the three core finding is good and easy to follow, some information currently in the appendix should be moved to the main paper. For example, the paper has a lot of results but it's missing a discussion of implications and more in general of the findings. My suggestion is to move some results less solid and conclusive (for example, the ones related to Chain Of Thought) in the appendix and move in the main manuscript some of the discussion.
- The tone describing the findings seems a little bit too optimistic. Indeed, GPT-4 shows a good behavioral alignment to human trust behavior and dynamics but the other LLMs often fail and this should be discussed in a more critical way.
questions: questions: - At page 3, the authors state that only GPT-4 is used to generate 53 types of personas. Why? Which was the outcome using other LLMs? Not realistic personas?
- At page 4, the authors state that "we select one BDI from personas giving an high amount of money and another BDI from those giving a low amount" ... this means just 1 BDI for condition? and why just 1? and how this one is selected?
- In Figure 6, the differences between condition (a) and condition (b) should be explained in the caption.
- The LLM agents' tendency to exhibit a higher level of trust towards women is an interesting result but it's not clear if this tendency is aligned to human tendencies. More specifically, are also human agents showing a similar tendency?
- Results in Figure 8 should be discussed. Often in the paper the authors mention biases towards race but there is no discussion of these results (just the Figure).
. In Figure 10 some results obtained by GPT-4 seem a little bit random (GPT-4 (4), GPT-4 (16), GPT-4 (14)) ... the authors should discuss them.
limitations: limitations: The authors should improve the discussion of when the LLM agents fails. While the result for GPT-4 are showing that it could be used as a simulation tool, the ones obtained for the other LLMs are more ambiguous and this should be discussed more in the paper.
paper_summary: paper_summary: This paper investigates whether Large Language Model agents can effectively simulate human trust behavior. The authors explore trust behaviors using the Trust Game and its variations, comparing the trust exhibited by these agents with that of humans. They find that GPT-4 shows a high degree of behavioral alignment with human participants. Additionally, they conduct various analysis experiments by altering player demographics, interaction objects, explicit instructions, and reasoning strategies.
strength_weakness: strength_weakness: - Trust games simplify real human trust behaviors and cannot fully represent trust behaviors. I suggest the authors rephrase the title to indicate a reasonable range, such as trust behaviors in trust games.
- The description of the dataset and setting is not sufficient, raising concerns about the soundness of the results. For example, how were the 53 personas generated by GPT-4 chosen? How to prove that they represent a broad spectrum of human personalities and demographics? Also, more details such as the statistics of pairs of agents in the game should be complemented, since the combination of similar personas and opposite personas may make a huge difference.
- The paper lacks analysis of the experimental results. Why GPT-4 can exhibit human-like factors while the small model with fewer parameters can/cannot? What specific capabilities of the models might affect the results? For example, the results of the prosocial factor may be caused by RLHF. This issue also exists in Sec.5. Overall, while the paper presents many findings, it does not sufficiently explain the underlying reasons for these results. 
- How each part of the human trust experiment was completed and what data was used should be briefly explained in the main body, even if the data is from previous work. This would help clarify the comparability of agent experiments and human experiments.
questions: questions: - Q1: I am curious about how much impact the prompt can have on the results, especially regarding the explanation of the trust game.
- Q2: Does the model exhibit this trusting behavior due to internal factors, or because it has the corresponding knowledge, such as common sense or domain-specific knowledge of how to play the game to maximize benefits as much as possible?
- Q3: Can the output of BDI be quantitatively analyzed in correlation with the decision results of the agents? This will be more convincing than just giving two cases in the current version.  
- Q4: Why LLM agents send more money to humans compared with agents? To an LLM agent, what’s the main difference it perceived between interacted humans/agents, such as the prompt and the returned response?
limitations: limitations: Yes. The paper has discussed the limitations.
paper_summary: paper_summary: This paper proposes a framework utilizing behavioral economic paradigms to investigate LLM's trust behaviors and compare them with human behaviors. This paper considers multiple LLMs (from small size to large and commercial size), as well as multiple tasks that circle the behavioral factors (Reciprocity Anticipation, Risk Perception, and Prosocial Preferences. The results show that with larger model sizes, LLMs are more likely to align with human data in these tasks (GPT-4 in the paper is the best model) in the aspects above. Finally, the authors also test the roles of demographic persona (e.g., gender), trustee's identity (LLM or humans), mandatory behavioral manipulation, and prompt engineering (e.g., CoT), and find those manipulations yield an impact on the behavioral patterns of LLMs.
strength_weakness: strength_weakness: - In Repeated Trust Game, as I found in the appendix, the prompts only provide the last round's feedback but do not provide a full behavioral history. This may not be a fair comparison to what humans face in the experiment. The authors also mention that in the last round, humans tend to choose not to pay back (this probably means human participants know it's the last round and maximize their rewards since they don't need to expect future payback). I would wonder if in the prompt, explicitly telling the LLMs it's the last round would generate a similar behavior to humans.

- The paper proposes a clear question *'Can LLMs really simulate human trust behaviors?'* and in the conclusion part the answer is already definite. But a concern is why this question is important. It is addressed in the paper *' Nevertheless, most previous research is based on one insufficiently validated hypothesis that LLM agents behave like humans in the simulation.'* If this paper only aims to propose a more comprehensive framework in evaluating LLM's behaviors (using trust behaviors as an example), this is good but not novel enough. Though there are implications in the appendix, most of them are not directly related to the value of this proposed question. For example, 'AI cooperation' or 'Human-AI' cooperation can use more direct paradigms to probe (rather than only trust games). Knowing whether LLM's trust behaviors are aligned with human trust behavior does not explicitly answer whether this LLM would have better cooperation with other agents or humans. That is to say, no evidence is shown in the paper that human trust behaviors are optimal in cooperation situations, and thus there are indeed gaps between this primary question and the implication areas.
questions: questions: - One question that comes to mind is: is it possible that the LLMs' training data already include those behavioral findings and probably even raw data somehow? This may be hard for exact tasks, but I may guess the result of LLM's preference to humans in the task or are more likely to invest more if the trustees are humans likely a projection of human preference hidden in the massive pre-training dataset. Indeed, there are research indicating that humans prefer to trust humans rather than machines in complex decision tasks. I would wonder whether this preference comes from the massive pre-training dataset, or in the RLHF phase. One possible way is to find some models with both versions and check whether their preferences to humans differ.
limitations: limitations: - As already indicated before, the limitation of the work may come from its novelty and social impact. I don't think whether LLM exhibits similar trust behavior to humans is necessarily important for enhancing the quality of agent cooperation or AI-human cooperation unless the authors could provide literature that investigating the cooperation behavior optimization and human trust behaviors are near optimal.
paper_summary: paper_summary: This paper studies the trust behaviors of LLM-based agents, which are important for agents to simulate humans. The authors focus on (1) how LLM-based express trust behaviors, and (2) the similarity/alignment between agent trust and human trust. They find that LLM-based agents generally exhibit trust behaviors in trust games, and agents with more parameters have high alignment in terms of trust behaviors. They also take further discussions on critical issues like biases.
strength_weakness: strength_weakness: 1. I think more experiments can be added to repeated trust games, because in the real world, the past behaviors (i.e., reputation) of trustees are important for trustors.
2. Trust games may be just a sub-field for trust behaviors. Some discussions and future works about them are expected. Could you please provide some insights?
3. What about the scenarios with more them two players? Such as the trust behaviors inside a group.
questions: questions: See above "Weaknesses". If the author could address my concerns, I'm willing to improve my rating.
limitations: limitations: The authors have discussed the limitations in the Appendix.

Comments: title: Thanks for your valuable feedback and the acknowledgement of our contributions!
Thanks for your constructive feedback and the acknowledgement of our contributions!
Thanks for your kind comment and the acknowledgement of our contributions!
Thanks for the kind comment!
Response to Reviewer TM2T on the implications and significance of our work
Could you please let us know your remaining questions or concerns?
comment: Dear Reviewer TM2T,

We would like to sincerely appreciate your valuable feedback and the acknowledgement of our contributions. We will provide more discussions on the connections between our findings and future applications in the revision. Thanks for your time and effort again!

The authors
I appreciate the authors for the patient rebuttal and additional literature provided. I think this work is comprehensive and technically sound, and I'd be happy if it appeared at NeurIPS this year. However, given the literature listed above, I don't see strong connections between this work and promising applications (which means this work will be fundamental work to future aspects). Therefore, I will maintain my current evaluation with a 'weak' accept.
Dear Reviewer QQ9h,

We are genuinely grateful for your constructive feedback and the acknowledgement of our contributions. We will follow your suggestions in the revision. Thanks for your time and effort again!

The authors
I read the answers to my comments and questions. Regarding the answer to my Question 1 I disagree with the authors. I think it would be interesting also evaluating experimental settings where other LLMs are used to generate personas. The results could be added in the Appendix. The reason is that the performance of GPT-4 and other models are quite different and I'm hypothesizing the same will happen for the generation of personas.

However, overall I'm satisfied with the work and I think it could be a valuable contribution to the conference.
Dear Reviewer 3T1u,

We sincerely appreciate your kind comment and the acknowledgement of our contributions. Thanks for your time and effort!

The authors
Dear Reviewer hfM5,

We are genuinely grateful for your kind comment. Thanks for your time and effort!

The authors
Dear Reviewer TM2T,

We sincerely appreciate your kind and detailed reply and are more than willing to provide more explanations as follows.

To start with, we acknowledge that aligning AI (such as LLMs) with humans does not necessarily imply that it makes AI better. **However, we would like to clarify that we did not claim that aligning AI’s trust behavior means better cooperation**. First, as discussed in Appendix B Implications, **trust has been long recognized as a vital component for effective cooperation in human society [1,2,3] and Multi-Agent Systems (MAS) [4,5]**. We envision that agent trust can also play an important role in facilitating effective and efficient cooperation of LLM agents. Second, we discover the behavioral alignment between LLM agents and humans regarding trust behavior, indicating that **these trust-dependent strategies in social science [1,2,3] that are effective in enhancing human cooperation are potentially also beneficial for cooperation in LLM agents**. 

It is worth noting that **our proposed behavioral alignment is distinct from value alignment, which is usually achieved through algorithms such as RLHF**. We discovered this phenomenon in existing LLMs and illustrated the broader implications in Appendix B and C. Specifically, our discovered behavioral alignment on trust behavior has broad implications on human simulation, agent cooperation and human-agent collaboration, **which reflect the significance of our discoveries**. 

For the implications on human simulation, it is worth emphasizing that our discoveries lay the foundation for simulating more complex human interactions and societal systems, since trust is one of the elemental behaviors in human interactions and plays an essential role in human society. Thus, **our findings provide empirical evidence for the applications of human simulation in various social science fields such as economics, politics, psychology, ecology and sociology [6,7,8] or role-playing agents as assistants, companions and mentors [9,10,11]**. 

Furthermore, in Section 5, **our work also conducts extensive investigation and sheds light on the intrinsic properties of agent trust beyond behavioral alignment**, including the demographic biases of agent trust, the preference of agent trust towards humans compared to agents, the impact of advanced reasoning strategies and external manipulations on agent trust. **These insights can inspire more future works to gain a deeper understanding of LLM agents’ decision making**.

We hope that we have fully addressed your concerns and are glad to provide more details if you have any more questions. Thanks again for your time and effort!

[1] Gareth R Jones and Jennifer M George. “The experience and evolution of trust: Implications for cooperation and teamwork”. Academy of management review, 23(3):531–546, 1998.

[2] Jeongbin Kim, Louis Putterman, and Xinyi Zhang. “Trust, beliefs and cooperation: Excavating a foundation of strong economies”. European Economic Review, 147:104166, 2022.

[3] Joseph Henrich and Michael Muthukrishna. “The origins and psychology of human cooperation. Annual Review of Psychology”, 72:207–240, 2021.

[4] Sarvapali D Ramchurn, Dong Huynh, and Nicholas R Jennings. “Trust in multi-agent systems”. The knowledge engineering review, 19(1):1–25, 2004.

[5] Chris Burnett, Timothy J. Norman, and Katia P. Sycara. “Trust decision-making in multi-agent systems”. In Toby Walsh (ed.), IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011,

[6] Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. “Large language models empowered agent-based modeling and simulation: A survey and perspectives”. arxiv 2023

[7] Benjamin S Manning, Kehang Zhu, and John J Horton. “Automated social science: Language models as scientist and subjects”. arxiv 2024

[8] Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. “Can large language models transform computational social science?” arxiv 2023

[9] Diyi Yang, Caleb Ziems, William Held, Omar Shaikh, Michael S Bernstein, and John Mitchell. “Social skill training with large language models” arxiv 2024

[10] Rania Abdelghani, Yen-Hsiang Wang, Xingdi Yuan, Tong Wang, Pauline Lucas, Hélène Sauzéon, and Pierre-Yves Oudeyer. “Gpt-3-driven pedagogical agents to train children’s curious question-asking skills”. International Journal of Artificial Intelligence in Education, pp. 1–36, 2023.

[11] Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. “From persona to personalization: A survey on role-playing language agents” arxiv 2024
Thanks to the authors for the comprehensive feedback. The authors did propose a comprehensive framework and have done much work in investigating the __behavioral alignment between LLM agents and humans regarding trust behavior.__ If the scope is just constrained with this statement, I will keep my original evaluation. 

The authors mentioned __']. The behavioral alignment between agent trust and human trust indicates that the human cooperation strategies could also be adopted in agent cooperation or human-agent collaboration to enhance the performance or efficiency and minimize the potential risk.'__. However, the authors did not show any empirical evidence from current studies or previous literature to support this claim. Aligning AI (particularly LLMs) to Humans is popular nowadays but does not essentially mean every aspect of alignment is necessary or better, especially given people don't know what is optimal. In other words, for this paper particularly, aligning AI's trust behaviors to humans does not necessarily mean better cooperation (since humans themselves may not cooperate optimally). To promote the understanding of alignment, research on optimality(whether from an individual or societal perspective) must be done before simply aligning A to B. Given this, though the authors have done comprehensive work on behavioral alignment and are technically solid, the contribution of the overall scope constrained this paper to get a higher score. Therefore, I will maintain my current evaluation.
Dear Reviewer 6WVz,

We sincerely appreciate your kind and quick reply! We believe we have fully addressed your concerns. Could you please let us know your remaining questions or concerns? We are more than willing to provide more details if you have any more questions. Thanks again for your time and effort!

The Authors
Thanks for the rebuttal by the authors. I would like to maintain my score of 5.