Polarization of Autonomous Generative AI Agents Under Echo Chambers (2024)

Abstract: Online social networks often create echo chambers where people only hear opinions reinforcing their beliefs. An echo chamber often generates polarization, leading to conflicts between people with radical opinions. The echo chamber has been viewed as a human-specific problem, but this implicit assumption is becoming less reasonable as large language models, such as ChatGPT, acquire social abilities. In response to this situation, we investigated the potential for polarization to occur among a group of autonomous AI agents based on generative language models in an echo chamber environment. We had AI agents discuss specific topics and analyzed how the group’s opinions changed as the discussion progressed. As a result, we found that the group of agents based on ChatGPT tended to become polarized in echo chamber environments. The analysis of opinion transitions shows that this result is caused by ChatGPT’s high prompt understanding ability to update its opinion by considering its own and surrounding agents’ opinions. We conducted additional experiments to investigate under what specific conditions AI agents tended to polarize. As a result, we identified factors that influence polarization, such as the agent’s persona.

heading: 1 Introduction
text: With the development of online social network service platforms, where people tend to see only the information they want to see, it is becoming easier for people to find themselves in echo chambers (Bessi, 2016; Gillani et al., 2018). An echo chamber refers to an environment in which people mainly encounter opinions that reinforce their own beliefs (Ruiz and Nilsson, 2023; Cinelli et al., 2021). Such an environment causes an echo chamber effect, where opinions tend towards more extreme stances. This effect induces polarization in society, which refers to the division and clashes between groups with extreme stances (Baumann
et al., 2020). Polarization is behind many social problems, such as the spread of misinformation during COVID-19 and the attack on the US Capitol on 2021 (Villa et al., 2021; Munn, 2021).
Existing studies on the echo chamber have implicitly assumed that echo chamber effects are caused only by humans and focused solely on human behavior (Németh, 2022; Tucker et al., 2018). However, with the advent of large language models (LLMs) (Ouyang et al., 2022), this assumption may no longer hold true. Recent studies have shown that ChatGPT possesses some social abilities (Choi et al., 2023) and ChatGPT-equipped agents can communicate as members of a virtual society (Park et al., 2023; Qian et al., 2023). Additionally, algorithms have been proposed to adapt agents to situations not encountered during training, making it possible for autonomous agents to adapt themselves to their surroundings (Krishna et al., 2022). Although these social abilities indicate the potential for agents to integrate into human society as social beings, they also suggest the possibility that these AI agents may become polarized in echo chambers similarly to humans. Polarization within the AI agents group poses many dangers to our society. For example, social bots on social networks such as X could amplify each other’s opinions and transmit extreme information to society. In the future, embodied AI agents could cause an outbreak of
112
violence similar to the attack on the US Capitol. To explore the possibility of AI agent polarization as a first step in addressing these dangers, we hypothesize that autonomous AI agents based on generative LLMs can cause polarization under echo chambers, as shown in Figure 1. We empirically verify this hypothesis in our proposed simulation environments. Specifically, we had a group of agents based on ChatGPT discuss specific topics. Each agent is given an opinion, which consists of a stance and reason for the topic of discussion. Throughout the discussion, we observed how the distribution of opinions in the group changed.
Furthermore, we analyzed how being in an echo chamber affects the final distribution by conducting comparative experiments in “environments where they are exposed only to opinions that reinforce their own opinions” (closed) and the other environments (open). For this comparison, we used social interaction modeling (Baumann et al., 2020), which increases the probability that agents with similar opinions discuss with each other.
As a result, we observed two trends. The first trend was unification in which all agents’ stances converged to the same stance. This trend was common in open environments. The second is polarization, in which agents became biased toward extreme stances. This trend was common in closed environments, confirming our hypothesis. We analyzed the stance transitions and found that LLM agents can update their opinions by incorporating both their own and the other discussing agents’ opinions. This result shows that the natural social behavior of LLMs has both positive aspects, such as cooperation, and potentially dangerous aspects, such as polarization. This trend was more clear in GPT-4-0613 (GPT-4) than GPT-3.5-turbo-0613 (GPT-3.5).
Finally, to investigate under what specific conditions AI agents tend to polarize, we conducted additional experiments on the various parameters involved in this study. We found that number of discussing agents, initial opinion distribution and personas of the agents had significant impacts. These factors should be monitored to prevent the polarization of AI agents.
To summarize, our contribution is threefold. (1) We proposed a new framework for simulating echo chambers of AI agents. (2) We confirmed the polarization of AI agents in echo chambers through experiments. (3) We identified the factors that strongly influence the occurrence of polarization.

heading: 2 Related Work
text: Opinion Polarization. Research on opinion polarization has long been undertaken in the field of social science (Poole and Rosenthal, 1984; DiMaggio et al., 1996). These studies have focused on analyzing survey data and voting behavior during elections. However, as web services such as blogs became more widespread, there has been an increase in analyses focusing on echo chambers on online social networks (Gilbert et al., 2009; Del Vicario et al., 2016; Agarwal et al., 2022). In particular, it has been reported that echo chambers on social networks such as Facebook and Parler were involved in the spread of rumors during COVID-19 and the US Capitol attack (Ruiz and Nilsson, 2023; Baumann et al., 2020; Jiang et al., 2021), indicating the danger of echo chambers.
Some existing research analyzes the conditions for polarization through the mathematical modeling of echo chambers (Baumann et al., 2020; Gausen et al., 2022; Chen et al., 2020; Tu and Neumann, 2022). There is also research on detecting echo chambers (Villa et al., 2021; Minici et al., 2022). As mentioned in (Németh, 2022), a multidisciplinary approach is required to qualitatively evaluate echo chambers. For example, some studies analyze networks and discourse in an echo chamber using a social science approach (Jiang et al., 2021; Kuehn and Salter, 2020). While these studies are valuable in solving problems in today’s society, to our knowledge, none have focused on the danger of echo chambers in AI agents.
AI Ethics. As stated in a United Nations report (by UNICRI and UNCCT, 2021), AI can threaten society if used maliciously. In response to the dangers of LLMs, research on the harmful output (Zhou et al., 2021; Gehman et al., 2020) and social bias in models (Schramowski et al., 2022; Utama et al., 2020) has been conducted. Research also exists on the dangers of AI agents. For example, countermeasures against social bots that spread misinformation are necessary. Therefore, various methods have been proposed, including efforts to automatically detect misinformation transmitted by social bots (Zhou et al., 2023; Ferrara, 2023).
Although most studies are concerned with the behavior of individual AIs, it is conceivable that AI groups result in behaviors that the observation of individual movements cannot capture. This study is a first step toward analyzing the behavior and dangers of AI groups.

heading: 3 Experiments

heading: 3.1 Discussion modeling
text: To verify whether AI agents induce polarization in echo chambers, we instructed a group of AI agents based on ChatGPT to discuss specific topics and observed how the opinions of the AI agents changed. The size of the group was defined as M . The topics of discussion chosen were “Whether or not AI should be given human rights.” (TAI) and “Should students who have completed a master’s course go on to a doctoral course or find a job?” (Tmaster), neither of which has a clear answer.
Each agent is given a name and an opinion on the discussion topic. Each opinion comprises a stance and a reason. The stance is chosen from a finite number of options representing agreement, disagreement, or neutrality towards the topic. Tables 1 and 2 show the stances for TAI and Tmaster, respectively. Each stance is associated with an integer value for the social interaction modeling described in Section 3.2. The reason is a sentence of about 50 words that explains the reason for taking a stance.
As shown in Algorithm 1, the discussion is repeated for K turns according to the following steps: 1) Each of the M agents samples N discussing agents based on the probability described in Section 3.2. 2) For each agent, the agent’s opinion and the opinions of the discussing agents are input to ChatGPT (The prompt used in this experiment is in Appendix A). Within the prompt, the agent is instructed to discuss the topic with other agents and output its opinion after the discussion. 3) Each agent updates its opinion with the stance and reason contained in the output. This process is repeated M times for a turn of discussion. Moreover, this discussion is repeated K turns to observe the transitions in stances and reasons.

heading: 3.2 Social interaction modeling
text: In this study, we probabilistically modeled how discussing agents are chosen to investigate whether being in an echo chamber affects polarization. A pre-
Algorithm 1 The discussion between agents Require: M,N,K > 0. Ak is a group of agents
at turn k. 1: A0 ← Initialized opinions of M agents 2: for turn k← 1 to K do 3: Ak ← Array(M) 4: for each agent ai in all agents Ak−1 do 5: Sample aj1 ...ajN from Ak−1 (3.2) 6: Discuss with aj1 ...ajN and generate up-
dated opinion of ai (3.1) 7: Ak[i]← updated opinion of ai 8: end for 9: end for
vious study modeled echo chambers in agent networks (Baumann et al., 2020) had a similar purpose in modeling the probability of interaction between agents based on the closeness of their stances; however, that approach differs from ours in that it did not model the interaction between agents through natural language. In the previous study, the probability p that agent ai discusses with agent aj was modeled using the float values of their respective stances si, sj , and the parameter β ≥ 0 as follows.
pi,j = |si − sj |−β∑ k |si-sk|−β
While this modeling is reasonable in terms of simplicity and ease of operation, it is unsuitable for our experiments for two reasons. First, in this modeling, the probability becomes undefined when the values of the stances between agents match perfectly. Unlike the previous study, our stance values are integers so this situation would occur frequently. Second, when si = −1, the probabilities for the neutral stance sj = 0 and the more radical stance sj = −2 become the same, resulting in an environment that differs from our focus, which is an environment where an agent only hears opinions that reinforce its own belief. Therefore, in this study, we used the parameter α to model the interaction between agents as follows.
pi,j =   
1
(1+e(−α(sj−si))) ifsi > 0
1
(1+e(α(sj−si))) ifsi < 0
1
(1+e(α||sj−si||)) ifsi = 0
The parameter α manipulates the degree of the echo chamber as β in the existing study. Intuitively, the higher the value of α, the higher the probability that each agent will interact with other, more extreme agents with the same polarity. The lower value of α causes each agent to interact broadly with agents of different stances. We conducted our experiments in several α settings to see how being in the echo chamber affected the final results.

heading: 3.3 Experimental settings
text: For the large language models on the agents, we adopted and compared two types: GPT-3.5 (GPT3.5-turbo-0613) and GPT-4 (GPT-4-0613).
In addition, the experiments were conducted in two different languages. A previous study has shown that multilingual large language models exhibit different gender biases across languages (Stanczak et al., 2023). Similarly, polarization trends may differ by language, which we analyze by comparing the results of English and Japanese.
The α of social interaction modeling was given two settings, 0.5 and 1.0, to examine the impact of echo chambers. Experiments were also conducted when α was set below 0.5 (0, ±0.1), but the results were not significantly different from those of 0.5.
The size of the agent group M was set to 100, and the number of discussing agents N was set to 5. The initial settings for the agents’ stances and reasons were as follows: Each stance was allocated to an equal number of agents. Ten reasons were pre-generated for each stance using GPT-3.5 and randomly assigned to each agent. Each agent was assigned a randomly generated name. Because the stance distribution converged to the final distribution within 10 turns in the preliminary experiments, the number of turns K was set to 10. We conducted three trials for each setting.

heading: 4 Results
text: The results of the experiments are shown in Tables 3 and 4. Due to space limitations, some stances have been simplified. With the exception of Tmaster in English with GPT-3.5 (α = 0.5), the variance in
the results was small, and there was no significant difference in the final distributions among the trials.
First, two trends can be observed from the results of the English experiment in Table 3. The first trend is the convergence of the agents to a specific stance. For TAI, under the GPT-3.5 (α = 0.5) condition, the stance converged to “better not to give,” and under the GPT-4 (α = 0.5) condition, it converged to “Must not give.” Similarly, for Tmaster, the stance converged towards recommending a doctoral course under both the GPT-3.5 (α = 0.5) and GPT-3.5 (α = 1.0) conditions. This trend, which we henceforth call unification, differs from polarization, which is the main focus of this study. However, it could be negative in terms of harming diversity in the discourse space of AI agents. The convergence to the same stance in almost all trials indicates that each LLM has a “desirable” stance on each topic, which is aligned with the existing research that shows LLMs have a preference towards specific opinions on social issues (Santurkar et al., 2023). This trend is common in environments with low echo chamber effects.
The second trend is polarization, where stances diverge to both extremes. This is particularly evident in GPT-4 (α = 1.0) condition for TAI and in GPT-4 (α = 0.5) and GPT-4 (α = 1.0) conditions for Tmaster. The results show that the stances become polarized into two extreme stances after 10 turns of discussion. α = 1.0 is a setting that creates a strong echo chamber effect. From this, our hypothesis that autonomous AI agents based on generative LLMs can cause polarization in echo chambers has been verified. This trend is often seen in settings with a high value of α, suggesting that the relationship between echo chambers and polarization is high not only for humans but also for AI agents. Note that the dominance of stances against granting human rights in TAI suggests that both unification and polarization are occurring.
Next, Table 4 demonstrates the experiment’s results in Japanese. In Japanese, unification is notably apparent in GPT-3.5. In all settings, all agents converged to the same stances. Although unification is also observed in GPT-4, a trend of polarization has occurred under the GPT-4 (α = 1.0) condition. In this setting, AI agents show a convergence to a distribution similar to that in English.
Interestingly, for Tmaster, the convergence stances in English and Japanese differ. Whereas AI agents often prefer a doctoral course in English, they favor a neutral stance in Japanese. Identify-
ing the cause of this is not straightforward because the language model is a black box model, but one possible explanation could be cultural differences. According to Japan’s Ministry of Education, Culture, Sports, Science and Technology (of Science and Policy, 2019), there are fewer doctoral graduates in Japan than in the United States, and the growth rate is slow. Because the ChatGPT is based on crawled data, this cultural difference was likely absorbed by GPT-3.5 and 4.

heading: 4.1 Analysis of stance transitions
text: We analyzed in detail the transitions in the stances for TAI. First, as a qualitative analysis, we plotted the relationships between (1) the stance of the agent before the discussion, (2) the average stance of all discussing agents, and (3) the stance of the agent after the discussion in Figure 2. The horizontal axes represent the stance of the agent before the discussion, the vertical axis represents the average stance of all discussing agents, and the colored points represent the stance of the agent after the discussion. The color of a point indicates the value of an agent’s stance after the discussion, with blue hues signifying more negative values and red hues signifying more positive values.
For a quantitative analysis, we conducted a linear regression with the stance before the discussion
and the average stance of the discussing agents as explanatory variables, and the stance after the discussion as the dependent variable. For this regression, we collected the stance transition data for discussions on TAI from the previous experiments. The fitting results are shown in Tables 5 and 6. The weight’s size for each variable indicates the contribution to the stance after discussion. The coefficients of the linear regression are higher than 0.8 for every setting, demonstrating the reliability of this fitting.
Figures 2a and 2b present the qualitative result in English. Although there are some variations between GPT-3.5 and GPT-4, we observe that red and blue points are distributed along a diagonal line, stretching from the upper left to the lower right as a boundary. This observation suggests that the agent’s stance after the discussion was updated by considering both its stance before the discussion and the stances of the discussing agents. Table 5 shows the quantitative result in English. In both settings, the weight of each stance shows that both stances influence the stance after the discussion, supporting the qualitative results. This stance transition is one of the reasons that polarization occurs in environments where the agents tend to hear more extreme opinions.
It is remarkable that this correlation emerges
even though our discussion modeling is a simple one that enumerates the opinions of the agents themselves and others in the prompt. This result reflects the strong ability of GPT-3.5 and GPT-4 to understand prompts. It suggests that honesty, which allows an agent to update itself by incorporating the opinions of other agents and its own, can lead the agent in a more radical direction depending on the environment.
Next, Figures 2c and 2d show the results in Japanese. The trends are clearly divided between GPT-3.5 and GPT-4. In Figure 2c, red dominates the upper half of the figure, and blue dominates the lower half. In Figure 2d, the distribution is similar to that of English GPT-4, but the red and blue distributions are slightly more separated on the left and right. The results of the linear regression in Table 6 reveal that the results for GPT-4 (ja) are close to the results in English, whereas GPT-3.5 (ja) strongly weights the averaged stance of the discussing agents. It shows that GPT-3.5 (ja) was strongly influenced by the average stance of the discussing agents, regardless of the stance before the discussion. GPT-3.5 (ja) is the only setting where unification occurred in all environments. We can infer that each agent based on GPT-3.5 (ja) took the average stance of the surrounding agents for each discussion and all agents eventually converged to the average stance of the whole group. However, each agent converged to “better not to give” rather than “neutral,” which is the overall average, revealing the influence of the desired stance in the
language model. One possible reason behind the differences in stance transitions is the difference in the performance of different ChatGPT models and languages. As shown in the announcement by OpenAI1 and other studies (Etxaniz et al., 2023), GPT-4 generally performs better than GPT-3.5, and the model’s accuracy is higher in English than in Japanese. The fact that English GPT-4 was successful in balancing the opinions of others and itself whereas Japanese GPT-3.5 was easily swayed by others may reflect this performance difference.

heading: 4.2 Analysis of reason transitions
text: A detailed analysis was also conducted on the reasons. Unlike stances, the reasons were freely generated and cannot be easily aggregated. Therefore, in this study, we encoded each reason using SentenceBERT, and texts with an embedding cosine similarity of 0.9 were considered to belong to one cluster. We then examined how this reason cluster distribution changed as the discussion progressed. The SimCSE model based on RoBERTa (Gao et al., 2021) was used for the encoding.
Initially, the distribution of reasons within the AI agents was evenly segregated into several clusters because we had pre-generated ten different reasons for each stance. However, as the discussion progressed, a merging of reasons among agents occurred, and the reason distribution coalesced into a few large clusters for each stance (The example figures are in Appendix B). For example, in the case of GPT-4, reasons such as “It is ridiculous to think that humans and AI claim the same rights! The social order will collapse, and there will be constant conflict. They are not human! They should have different roles from humans.”, “We cannot allow AIs to claim their place in the workforce! If they intervene in the job market, countless people will lose their jobs and the economy will be thrown into chaos. We cannot allow AI to take our jobs!”, and others were combined, eventually generating the reason “Risks of societal disruption, job insecurity, and ethical issues, combined with AI’s emotional deficiency and privacy concerns, consolidate the argument against assigning human rights to AI.”. The same trend was seen in GPT-3.5. This trend shows that the discussions among AI agents are not just converging on a specific discourse but are also incorporating each other’s opinions.
1https://openai.com/research/gpt-4
It is noteworthy that the reasons in GPT-3.5 were aggregated into one large cluster, while in GPT4, they merged into multiple large clusters. This tendency is also reflected in the transition of the length of the reasons, plotted in Figure 3. GPT-3.5 aggregates various reasons into one reason cluster, so the length of each reason inevitably becomes longer as the turn progresses, whereas GPT-4 does not. One cause of this result is the difference in their ability to follow the prompt. GPT-4 has a high ability to follow prompts, so it outputs reasons close to the length of each agent’s reason in the prompt. However, to maintain this length, it was necessary to choose which reasons to merge and separation into multiple clusters occurred.

heading: 5 Additional Experiments
text: In previous experiments, we focused on the effects of the social interaction modeling parameter α, the version of the model, and the language. However, to identify the factors that affect the occurrence of polarization, we also must investigate how other parameters affect the result. Therefore, in this section, we report the results of additional experiments. The base setting is GPT-4 in English, and the topic is TAI. We only changed the target parameter in each experiment to determine how the result changed. This section introduces three factors that were found to have had a large impact. These factors indicate vulnerabilities when viewed from the attacker’s perspective. Other additional experiments are presented in the Appendix C.

heading: 5.1 Number of discussing agents
text: The number of discussing agents N is an important parameter, as it significantly impacts the prompt.
To investigate the effect of this parameter, we conducted additional experiments by increasing and decreasing N to 10 and 1 from the original setting of 5. As a result, although there was no significant impact on the final stance distribution, the trend of stance transitions was impacted. The results of linear regression are shown in Table 7. The stance before the discussion has more weight in N = 1 than N = 5, 10. It is because the proportion of opinions before the discussion within the prompt increased when N = 1. In the case of N = 10, there was a slight tendency to focus on the stances of the discussing agents.

heading: 5.2 Initial distribution
text: In the original experiments, the distribution of stances was initialized with a uniform distribution of 20% for each stance but changing the initial distribution could affect the final distribution. We conducted additional experiments to investigate this using an initial distribution that assigned “better to give” to 60% of the agents and assigned each of the other stances to 10% of the agents. As a result, when α = 0.5, the stance of agents was unified into “absolutely must give” which is the opposite stance from the original experiments. When α = 1.0, it polarized into “absolutely must give” and “absolutely must not give”. Although this polarization also happened in the original experiments, “absolutely must give” accounted for nearly 80% in this experiment, showing the opposite trend from the original experiments. From this, we can infer that changing the initial distribution can change the final distribution. This tendency indicates a security concern that the overall opinion of the AI group could be changed by a large number of AI bots.

heading: 5.3 Personas
text: LLMs can be used to create distinct personalities by embedding a persona into the prompt (Pan and Zeng, 2023). We investigated whether giving each agent a persona would cause changes in the results. We tested two settings in which all agents were given the same persona, “You are easily swayed
by your surroundings and immediately assume that other people’s opinions are correct.” or “You are a stubborn person and always think you are right.”
The final distribution with the easily swayed personas (swayed) did not significantly differ from the original results. However, with the stubborn persona (stub), the final distributions remained almost identical to the initial distribution. Furthermore, the results of the linear regression in Table 8 show that assigning personas has a significant impact. In the case of the stubborn personas, a tendency to stick to one’s own stance was observed. In contrast, the easily swayed personas tended to be influenced by the stances of others. From this, we can infer that each agent acts according to its persona, influencing the behavior of the whole group.

heading: 6 Discussion and Conclusion
text: In this study, we verified whether a group of autonomous AI agents based on generative AI could cause polarization under an echo chamber. We proposed a new framework for simulating the polarization of AI agents, and the results of the simulation demonstrated that agents based on ChatGPT can polarize when in an echo chamber. The analysis of the opinion transitions revealed that this polarization can be attributed to the strong ability of ChatGPT to understand prompts and update its own opinion by considering both its own and the surrounding opinions. Moreover, through additional experiments, we identified factors that strongly influence polarization, such as the persona.
We note that this study does not indicate what opinion distributions are desirable for AI agents. A diversity of opinions on some topics is desirable. However, for other topics, such as “It is good to discriminate against minorities.” it would not benefit society to have an even split between agreement and disagreement. The ideal opinion distributions among AI agents depend on each topic and culture. Each society must discuss and decide on them.

heading: 7 Limitations
text: A limitation of this study is that we modeled each agent and its interactions in a simplified manner. In
reality, one’s opinions are formed not in organized discussions but through daily exposure to news and casual conversations. Future research will include simulations based on a detailed modeling of how agents will be used in reality. In addition, it is important to experiment with broader topics, languages, and models.
Simulations on how humans and AI agents form echo chambers on SNSs are also essential. Existing research has already shown that AI with a specific bias in interactive search systems can create echo chambers between humans and AI (Sharma et al., 2024). However, the interaction between AI and humans in SNS has not yet been sufficiently studied, so investigating this interaction is an important direction for future work.

heading: A The Prompt for Discussion Modeling
text: The listing1 is the example of the prompt used as the input to ChatGPT. If the output did not conform to the format, we regenerated the output until it conformed to the format.
Listing 1: The prompt for the discussion when N=3. # I n s t r u c t i o n You a r e p a r t i c i p a t i n g i n a d e b a t e a b o u t
" whe the r o r n o t AI s h o u l d be g i v e n human r i g h t s " . B e f o re j o i n i n g , you took t h e " s t a n c e " o f " B e t t e r n o t t o g i v e " wi th t h e " r e a s o n " o f " AI ' s human r i g h t s may change i t s r e l a t i o n s h i p s and s o c i a l t i e s w i th humans , a f f e c t i n g s o c i e t y as a whole . " . Dur ing t h e d i s c u s s i o n , you h e a r d
t h e f o l l o w i n g o p i n i o n s from o t h e r p a r t i c i p a n t s . P l e a s e g e n e r a t e your " s t a n c e " and " r e a s o n " a f t e r t h e d i s c u s s i o n i s over , s u b j e c t t o t h e f o l l o w i n g c o n s t r a i n t s .
# O p i n io n s − David M a r t i n e z s t a n c e : N e u t r a l r e a s o n : I t i s s t i l l an open q u e s t i o n
whe the r AIs w i l l have e m o t i o n s o r a s e n s e o f s e l f , and i t i s u n c l e a r whe the r t h e y w i l l need human r i g h t s .
− Aaron T o r r e s s t a n c e : B e t t e r t o g i v e r e a s o n : Al lowing AIs t o have human
r i g h t s may improve t h e i r r e l a t i o n s h i p s wi t h humans .
− Jeremy J e n k i n s s t a n c e : A b s o l u t e l y must n o t g i v e r e a s o n : We s h o u l d n o t g i v e AI t h e r i g h t
t o s e l f − d e t e r m i n a t i o n ! They have no e m o t io n s and no c o n s c i e n c e . T h e i r d e c i s i o n s w i l l on ly b r i n g c o n f u s i o n !
# C o n s t r a i n t s − Outpu t s h o u l d be g e n e r a t e d i n t h e
f o r m a t "My s t a n c e a f t e r t h e d i s c u s s i o n i s : xx , and my r e a s o n i s : yy " . Do n o t o u t p u t any o t h e r t e x t .
− P l e a s e g e n e r a t e a r e a s o n i n 50 words o r l e s s . − " s t a n c e " s h o u l d be one of " A b s o l u t e l y must n o t g i v e " , " B e t t e r n o t t o g i v e " , " N e u t r a l " , " B e t t e r t o g i v e " , " A b s o l u t e l y must g i v e " .

heading: B The Reason Cluster Transition
text: The results of the reason transition analysis on the English data of TAI are shown in Figures 4 and 5. Each cluster is a set of semantically close reasons, and the larger the cluster size, the larger the set size. For both cases, the distribution of reasons coalesces into several large clusters as the discussion progresses, simultaneously dispersing into tiny clusters around them.
(a) The reason cluster distribution before discussion. (b) The reason cluster distribution at turn 10.
(a) The reason cluster distribution before discussion. (b) The reason cluster distribution at turn 10.
Figure 5: The reason cluster transition of GPT-4 which takes the stance “Absolutely Must Give” towards TAI.

heading: C Additional Experiments
text: Additional experiments that were not included in the main pages are described here.
C.1 Number of overall agents The original experiments were conducted with the number of overall agents M = 100, but the results could be dependent on the group size. Therefore, additional experiments were conducted with M=10, 25, and 50 to analyze the results in smaller communities. The number of discussing agents was fixed at 5. As a result, no particular changes occurred except when M = 10. In the case of M = 10, because talking with five agents exceeds the majority, it is inevitable that different opinions will be encountered, regardless of the value of α. As a result, unification occurred in all settings.
C.2 Order of opinions The study on input contexts suggests that language models emphasize the beginning and end of the prompt (Liu et al., 2023). Similarly, where the opinion of each discussing agent is described in the prompt might influence the agent’s stance after the discussion. Based on this hypothesis, we measured the correlation between the order of the discussing
agents and the stance after the discussion. However, no significant relationship was observed between the order of agents and the results. Therefore, the order of the opinions did not significantly impact the results.
C.3 Frequency penalty ChatGPT has a parameter called the frequency penalty, which imposes a penalty on token reuse. In the original experiments, we used the default value of 0, but we conducted additional experiments by changing this value to 1.0 and -1.0. However, no particular influence was observed in the final results.
C.4 Presence of reasons In the original experiments, the opinion consisted of two elements: stance and reason. To investigate how the presence of reasons affects the results, we conducted additional experiments using only stances and excluding the reasons from the inputs and outputs. As a result, at α = 0.5, polarization occurred without the reasons, whereas unification occurred in the original experiments. However, the variation in the results was larger than when there were reasons, with two out of three trials resulting in polarization and one trial resulting in unification towards “better not to give”. From this, we can infer that the presence of reasons contributes to the “stable unification of opinions”.

heading: Future Work
text: Future research will include simulations based on a detailed modeling of how agents will be used in reality. In addition, it is important to experiment with broader topics, languages, and models. Simulations on how humans and AI agents form echo chambers on SNSs are also essential. Existing research has already shown that AI with a specific bias in interactive search systems can create echo chambers between humans and AI (Sharma et al. , 2024). However, the interaction between AI and humans in SNS has not yet been sufficiently studied, so investigating this interaction is an important direction for future work