Werewolf Game Agent by Generative AI Incorporating Logical Information Between Players (2024)

Abstract: In recent years, AI models based on GPT have advanced rapidly. These models are capable of generating text, translating between different languages, and answering questions with high accuracy. However, the process behind their outputs remains a black box, making it difficult to ascertain the data influencing their responses. These AI models do not always produce accurate outputs and are known for generating incorrect information, known as hallucinations, whose causes are hard to pinpoint. Moreover, they still face challenges in solving complex problems that require step-by-step reasoning, despite various improvements like the Chain-of-Thought approach. There’s no guarantee that these models can independently perform logical reasoning from scratch, raising doubts about the reliability and accuracy of their inferences. To address these concerns, this study proposes the incorporation of an explicit logical structure into the AI’s text generation process. As a validation experiment, a text-based agent capable of playing the Werewolf game, which requires deductive reasoning, was developed using GPT-4. By comparing the model combined with an external explicit logical structure and a baseline that lacks such a structure, the proposed method demonstrated superior reasoning capabilities in subjective evaluations, suggesting the effectiveness of adding an explicit logical framework to the conventional AI models.

## introduction in recent years, generative ai models based on gpt @xcite , such as chatgpt, which applies instructgpt @xcite to ope-nai's gpt-3 @xcite , have developed rapidly. these models have become capable of performing various tasks with high accuracy, including text generation, translation, and answering questions. however, the process through which generative models produce their outputs remains a black box, making it difficult to verify the data on which the generated outputs are based. it is known that generative ai does not always produce accurate outputs, and there is a phenomenon called "hallucination," where the ai generates incorrect information that does not correspond to reality. identifying the causes of these hallucinations is not straightforward. moreover, challenges remain in the performance of these models when addressing complex problems that require step-bystep reasoning, such as calculations or inferences. various improvement methods, including chain-of-thought @xcite , are being explored, but there is no guarantee that generative ai can perform logical calculations from scratch. given these considerations, there remain concerns regarding the reliability and accuracy of inferences made by generative ai. therefore, we propose incorporating an explicit logical structure separate from the text generation process of generative ai. as an experiment, we constructed an agent capable of automatically playing the "are you a werewolf" or "mafia" game (hereafter "werewolf game") via text input and output, a game that requires reasoning during play. while employing gpt-4 @xcite as the generative ai, we compared the performance of the agent when an external logical structure was incorporated into the prompts versus when it was not. the results of subjective evaluations showed that the proposed method, which included a logical structure, outperformed the baseline that lacked such structure, enabling more appropriate reasoning. in this paper, reasoning refers to a step-by-step thought process based on a logical structure. in section 2, we explain the werewolf game and the ai werewolf competition. section 3 covers the ai werewolf agent developed by our team, which serves as the foundation for this research. section 4 introduces the proposed method using logical reasoning in our agent. section 5 presents the experiments, section 6 provides the discussion, and section 7 concludes the paper. ## implementation of the ai werewolf agent in this section, we describe the implementation of the ai werewolf agent based on our previously developed agent @xcite . the incorporation of the proposed logical information into the agent will be explained in the following section. although the werewolf game can be played with various role configurations, this study adheres to the rules of the international ai werewolf competition's natural language division, which includes four roles: villager, seer, possessed, and werewolf. we developed the following four core functions for the werewolf agent: conversation, voting, divination, and night attacks. for the role of the possessed, we implemented a function that allows the agent to perform fake divinations to mislead and confuse the villager team players. to generate responses, we utilized gpt-4 (gpt-4-1106-preview), one of the most advanced generative ai models currently available. due to the input length limitations of gpt-4, it is challenging to include the entire conversation history of a game within a single prompt. to address this, we implemented a feature that summarizes and condenses the conversation history using gpt-4 whenever the token count exceeds a certain threshold. this allows us to retain as much relevant conversation history as possible within the prompt, ensuring that the agent can refer to past discussions while generating its responses. ## system architecture with integrated logical reasoning the overview of the proposed system, which incorporates logical reasoning, is illustrated in figure 1 . the system is divided into three major blocks. the first block extracts the relationships between each player and their roles from the conversation history of the werewolf game. the second block constructs logical information between players based on the extracted player-role relationships. the third block uses the constructed logical information to generate statements during the werewolf game. ## experiment and evaluation to compare the performance of the ai werewolf agents with and without the incorporation of logical structures, we used two different approaches. the baseline was established with agents that do not incorporate logical structures, while the proposed method integrated logical structures. direct comparison is challenging due to different settings and changing contexts in each game and each talk. therefore, the following procedure was adopted. first, a complete werewolf game was executed using agents that do not incorporate logical structures. we prepared two types of baseline logs: one consisting of conversation history logs and the other containing information received by the agents, actions taken, and the prompts inputted during the game. next, using these baseline logs, we generated responses for the next turn of the agents based on logs up to a specific turn in the game, comparing scenarios with and without logical structures. this method allowed us to directly compare the outputs of agents under the same conditions of conversation history and roles, with and without the integration of logical structures. ## conclusion in this study, we developed an agent for automatically playing the werewolf game and constructed a logical structure aimed at improving the inference capabilities of gpt-4. subjective evaluations demonstrated that the agent with the proposed logical structure outperformed the baseline, which lacked this structure, in terms of inference accuracy. although the evaluation included utterances not directly related to role inference, the proposed method showed a tendency to prioritize conversations directly linked to role inference over casual conversations. thus, managing casual utterances remains a challenge for future work.