Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games (2025)

Abstract: LLMs are used predominantly in synchronous communication, where a human user and a model communicate in alternating turns. In contrast, many real-world settings are asynchronous. For example, in group chats, online team meetings, or social games, there is no inherent notion of turns. In this work, we develop an adaptive asynchronous LLM agent consisting of two modules: a generator that decides what to say, and a scheduler that decides when to say it. To evaluate our agent, we collect a unique dataset of online Mafia games, where our agent plays with human participants. Overall, our agent performs on par with human players, both in game performance metrics and in its ability to blend in with the other human players. Our analysis shows that the agent’s behavior in deciding when to speak closely mirrors human patterns, although differences emerge in message content. We make all of our code and data publicly available. This work paves the way for integration of LLMs into realistic human group settings, from assistance in team discussions to educational and professional environments where complex social dynamics must be navigated.

## introduction recent advances in llms have enabled their use in various applications and domains, the majority of which focus on synchronous settings -where communication is done in turns, often alternating between a human user and a model. in contrast, much of real-world communication is done asynchronously, allowing interlocutors to communicate at arbitrary times. in such communication, each participant needs not only to decide what to say, but also when to speak, allowing participants to adopt different strategies, e.g., being very talkative, or alternatively staying quiet. despite its prevalence in real-world interaction, to the best of our knowledge, there is no prior work that targets group asynchronous communication in the context of llms. instead, we find that the models developed for social interaction and other naturally-asynchronous settings, are modeled as involving predefined turns in which the model can interact with the environment (e.g., @xcite ; @xcite ). in this work, we develop an llm-based agent for such asyncronous multi-party environments, applicable in a wide range of real-world settings, including group chats, online team meetings, or social games. to address the problem of asynchronous group communication, we develop a two stage approach, splitting the decisions of what to say and when to say it into two separate tasks, each handled by an llm. our agent, described in section 2, orchestrates a two-stage call to an llm. first, an llm is prompted to decide whether to speak with a scheduler prompt. this prompt consists of the game's current status (e.g., chat history), along with an instruction which is amended according to previous decisions that the scheduler has made: if it has been quiet relative to the other participants, the prompt may urge the llm to be more talkative, while if it chooses to speak more than other participants, the instruction is amended to signal it should speak only if necessary. if the scheduler decides to speak, a second call is made to prompt an llm to decide what to say given previous messages. to evaluate our agent, we collect a novel mafia game dataset -in an online asynchronous social setting (section 3). mafia is a social game that includes deception and voting out suspicious players. as such, deciding when to speak is a crucial part of the player's strategy. for example, speaking too often or too little may seem suspicious. furthermore, winning the game serves as a proxy for successful communication. our dataset consists of mafia games including both human players and an llm agent whose identity is unknown to the human players (section 4), comprising 33 games and over 3.5k messages with over 90 unique human players. finally, we analyze the performance of our asynchronous agent in the mafia game (section 5) through various metrics, including game performance, objective timing measurements, and subjective human assessment through participant questionnaires. we find that our agent aligns with human players in message timing, message quantity, and winning rates. furthermore, 70b llm agents are hard to identify, and receive high scores for similarity to human behavior and timing of messages. however, we also find notable differences in message content, where the agent's messages are longer and can be distinguishable from human players by learned classifiers. in conclusion, our contributions are twofold. first, we address the problem of asynchronous group communication by proposing an asynchronous agent capable of effective real-time decision-making about when to speak, and show that it resembles humans when choosing when to speak. second, we publish a new dataset for mafia games, enabling future study of asynchronous communication. by focusing on a realistic, dynamic setting with natural asynchrony, our work lays the foundation for future research into multi-agent communication that more closely resembles human interaction. ## asynchronous agent we propose an agent designed for asynchronous conversations, consisting of two modules: a scheduler, deciding whether to post a message at a given moment, and a generator, which composes the message, both of which consist of backbone llm calls, which our agent orchestrates. the complete architecture is presented in figure 2 . scheduler module. the agent periodically runs a scheduler module to decide whether to intervene in a discussion at a certain point in time. this is achieved by prompting an llm to make a binary decision regarding whether it is now a good time to talk. the scheduler prompt consists of the following components (see orange box in figure 2 ): (1) persona traits, detailing different personality traits (e.g., extrovert versus introvert); (2) agent goals, defining what the agent is trying to achieve in the conversation (e.g., in a social game setting, this can consist of its rules and the agent role in the game); (3) the current time, an important aspect in asynchronous communication, allowing the agent to take the timing of its messages into account; and (4) the history of the conversation so far, along with the timing of each message. 2 finally, we experiment with dynamic scheduler prompting (green box in figure 2 ). to make sure the scheduler maintains a balance between being overly talkative and too quiet, we dynamically change the scheduler prompt based on the agent's message rate. when the rate of messages by the agent is lower than 1 /n (where n is the number of active participants in the conversation), we use a prompt which encourages it to speak more. ## asynchronous test case: the game of mafia evaluation of asynchronous communication is challenging due to the lack of labeled data distinguishing correct from incorrect message timing. therefore, we choose to set our evaluation of asynchrony modeling for llms in a game setting. social games serve as a valuable testbed, for several reasons. games give each participant an objective, while winning the game can serve as a proxy metric of whether the communication was successful and they set the conversation under a frame of rules, where each participant needs to use the communication to advance to their target. game over! game starts: everyone learns their role, mafia members learn each other's identities daytime everyone discusses who to vote for when phase is over: everyone votes for someone to eliminate a player is eliminated, their role is revealed nighttime only mafia can communicate when phase is over: mafia votes for a bystander to eliminate a bystander is eliminated, their role is revealed game ends: all roles are revealed game not over yet game not over yet game over! figure 3: flow chart of mafia's rules. the game continues until one of the teams achieves their objective: the mafia's goal is to outnumber the bystanders, and the bystanders' goal is to vote out all mafia. 4 we choose the mafia game for several reasons. first, it can be based solely on textual interaction, which allows llms to play together with human players. second, it requires collaboration under uncertainty, making communication between participants a fundamental aspect of the game. third, it centers around suspicion of other players, so both extreme strategies of constantly speaking or not speaking at all can be seen as suspicious. therefore, the timing of communication is crucial for the player's success. ## analysis we now analyze the performance of the asyncrounous llm agent relative to human players in various facets, including message timing and quantity, its content, and the agent's win-loss ratio.. message timing and quantity are similar to human behavior, albeit with reduced variance. the agent sends longer messages. as can be seen in ## related work here we explore related work in the fields of multiparty communication with llm, as well the development of models for social games. as will be highlighted below, to the best of our knowledge, our work is the first to explore asynchronous communication in these contexts. ## future work our study lays the groundwork for modeling asynchronous communication in llm agents, but much remains to be explored. one promising direction is to explore alternative asynchrony strategies beyond our two-stage prompting approach used here. for example, the generator could first generate a candidate message and then use the scheduler to decide whether to send it, inspired by human behavior of considering whether a thought is appropriate to express. another potential strategy involves finetuning the llm to output a special "<@xmath0, offering a more natural integration of silence as a communicative act. furthermore, these strategies can be compared with a turn-based inspired strategy, where the llm is prompted to speak every n new messages, with n set to the number of active players. this line of work can be extended to more accessible and scalable environments. our framework of asynchrony can be augmented to existing platforms, such as textarena @xcite . it would enable broader data collection from human-llm interaction in a wide variety of games, leveraging the natural engagement of players who participate for amusement, curiosity or challenge. such a platform could serve as a testbed for more complex group dynamics and open-ended behaviors, facilitating deeper research into llm social reasoning, coordination, and deception in asynchronous multi-agent contexts. ## discussion and conclusion we introduce a novel approach for enabling llm agents to participate in asynchronous multi-party communication, where the agent must decide not only what to say, but also when to say it. we implement a two-stage prompting framework, and integrated it in a realistic, dynamic environment -the social deduction game mafia, alongside human players. our agent demonstrated competitive game performance, blended into human groups, and exhibited timing patterns that align with human behavior, despite differences in message content. these results underscore the feasibility and value of incorporating asynchrony into the communication capabilities of llms. by moving beyond turnbased interactions, our agent more closely mirrors real-world conversational dynamics, where timing, silence, and strategy are essential components of communication. this opens the door to integrating llms in collaborative settings such as online team meetings, classroom discussions, and support groups, where agents must navigate social nuance and determine when their contributions are helpful, disruptive, or unnecessary. ultimately, modeling asynchrony equips llms with a richer understanding of human interaction, enabling more natural, context-aware participation in group settings. we hope that this work encourages further exploration of asynchronous llms and their integration into social environments.