LLM -Based Explicit Models of Opponents for Multi-Agent Games (2025)

Abstract: In multi-agent scenarios, the ability to anticipate and respond to opponents is essential, particularly in environments involving adversarial and collaborative interactions. In this paper, we introduce Explicit Models of Opponents (EMO) based on Large Language Models (LLMs), enabling agents to better predict and adapt to diverse, dynamic multi-agent interactions. Unlike traditional methods that often simplify multi-agent interactions using a single opponent model, EMO constructs an individual model for each opponent and aligns these models working in synergy through a bi-level feedback-refinement framework. We test EMO alongside several reasoning methods in multi-player deduction games, where agents must infer hidden information about their opponents. The results show that EMO significantly enhances agents’ decision-making, outperforming traditional single-model approaches. Our findings demonstrate that EMO can be a powerful tool for enhancing LLM-based agents in complex multi-agent systems.

## introduction autonomous agents, computational entities capable of operating independently in dynamic environments, have become integral to a wide range of applications @xcite . their core attributes-autonomy, perception, intelligence, social ability, and learning capacity-enable them to make decisions and take actions in pursuit of their objectives without external intervention @xcite . the rise of large language models (llms) has significantly expanded the capabilities of autonomous agents, allowing for more advanced reasoning, collaboration, and problemsolving across diverse domains @xcite . by integrating the cognitive abilities of llms, agents are now able to tackle more complex tasks with greater adaptability and effectiveness. * † correspondence to zongqing.lu@pku.edu.cn however, one critical aspect of agents, especially in environments with both adversarial and cooperative interactions, is the ability to model the behavior of other agents (we call them 'opponents' throughout the paper) @xcite . traditional approaches often simplify this by treating other agents as a single agent rather than as distinct entities with their own intentions and strategies @xcite . this limitation restricts agents from fully anticipating and responding to the behavior of others, particularly in high-stakes scenarios where understanding the goals and actions of opponents is essential for success @xcite . to address this, we introduce explicit models of opponents (emo), a novel framework that enables llm-based agents to explicitly and effectively model the behavior of their opponents. by doing so, agents can make more informed decisions, predict opponents' actions more accurately, and ultimately improve their performance in both cooperative and competitive environments. emo goes beyond traditional learning-based approaches by leveraging the inherent reasoning capabilities of pre-trained llms, allowing for more adaptive and dynamic responses to varying opponent strategies. the emo framework employs a bi-level feedback-refinement process, wherein opponents are modeled individually through atomic opponent models, which are continuously improved via feedback mechanisms. these individual models are then validated and refined globally by a central validator, ensuring that the predictions align with the broader context of the game or task. this process results in a more comprehensive understanding of the multi-agent environment, leading to better collaboration and conflict resolution among agents. to evaluate the effectiveness of emo, we conduct experiments in multi-player deduction games, including who is the undercover (witu) and avalon. these games present dynamic, imperfect-information scenarios where agents must infer hidden information about their opponents. the results of our experiments show that emo significantly enhances agent performance compared to existing methods, particularly in scenarios requiring complex reasoning and strategic adaptation. our key contributions are summarized as follows: 1) we propose emo, a novel framework for llm-based opponent modeling via explicitly constructing individual models for opponents and iterative feedback-refinement mechanisms; 2) we show emo has the potential to significantly improve decision-making, adaptability, and overall performance in dynamic, multi-agent games; 3) the findings suggest that explicit opponent modeling can be a key factor in advancing the capabilities of llm-based agents. theory of mind (tom) has been incorporated into llm-based agents @xcite @xcite to model the beliefs, desires, and intentions of others, leading to more sophisticated and adaptive interactions. muma-tom @xcite focuses on multi-modal inputs to help agents infer and adapt to the mental states of others, while suspicion-agent @xcite applies tom in imperfect information games, using llms to predict and influence opponents' behaviors. agent-pro @xcite employs self-belief and world-belief(other agents) to guide decisions. however, ensuring consistency and accuracy in belief inference remains a challenge. research on symbolic representations @xcite proposes using explicit graphs to track agents' mental states for more precise reasoning. meanwhile, studies on belief tracking systems @xcite explore how maintaining dynamic belief states improves decision-making. these studies form the foundation for llm-based tom, aiming to enhance agent collaboration, competition, and decision-making @xcite . unlike tom methods, our approach takes a different path by explicitly modeling multiple opponents, offering a more direct way to predict their actions. opponent modeling is a key technique in multiagent systems, where agents must predict and adapt to the behaviors and strategies of others. som @xcite allows agents to use their own policies to infer the goals of others, enhancing decision-making in both cooperative and competitive scenarios. l2e @xcite enables agents to quickly adapt to unknown opponents without requiring extensive data. m-fos @xcite and tp-mcts @xcite shape and predict opponents' actions without needing explicit knowledge of their learning processes, while mol @xcite and mbom @xcite simulate and adapt to opponents' evolving strategies through recursive reasoning and equilibrium prediction. additionally, full douzero+ @xcite employs opponent modeling in imperfect information games, while tdom @xcite dynamically models opponents' changing policies to handle non-stationarity in mixed environments. unlike these methods, we introduce llm-based explicit models of multiple opponents, offering an explainable way for opponent modeling. multi-agent debate (mad) involves multiple agents engaging in discussions to collaborathe central validator participates in steps 2 and 3, and later summarizes information from all atomic opponent models to provide it to the reasoning module for decision-making. liang et al. (2023) proposed a mad framework to enhance divergent thinking in llms and address reasoning degeneration by involving multiple agents, while liu et al. (2024) introduced group debate to improve the scalability of multi-agent debates. du et al. (2023) showed that multi-agent debate significantly improves factuality and reasoning and reduces hallucinations by cross-examining others' reasoning. wang et al. (2024) introduced a knowledge-enhanced mad framework that incorporates external knowledge to solve the problem, where agents stick to incorrect viewpoints. while mad has shown promising results in improving reasoning and factual accuracy, its consideration of other agents relies on methods like cot. this limits its effectiveness in dynamic environments. our approach aims to address this limitation by equipping agents with the ability to predict the actions of other agents in dynamic environments. ## conclusion in this paper, we introduced emo, which enhances the reasoning capabilities of llm-based agents in multi-agent games. emo leverages a bi-level feedback-refinement framework, enabling agents to better anticipate and adapt to the behavior of individual opponents. our experiments in "who is the undercover" and avalon demonstrate that emo significantly improves the performance of agents in both adversarial and cooperative roles. by explicitly modeling each opponent, emo provides more strategic insights, leading to higher performance compared to traditional single-model approaches. these results highlight the potential of explicit opponent modeling to advance llm-based agents' capabilities in complex multi-agent environments. future work may explore further optimizations in emo's architecture and its application to more diverse and dynamic scenarios.