B attle A gent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024)

Abstract: This paper presents BattleAgent , a detailed emulation demonstration system that combines the Large Vision-Language Model (VLM) and Multi-Agent System (MAS). This novel system aims to emulate complex dynamic interactions among multiple agents, as well as between agents and their environments, over a period of time. The emulation showcases the current capabilities of agents, featuring fine-grained multi-modal interactions between agents and landscapes. It develops customizable agent structures to meet specific situational requirements, for example, a variety of battle-related activities like scouting and trench digging. These components collaborate to recreate historical events in a lively and comprehensive manner. This methodology holds the potential to substantially improve visualization of historical events and deepen our understanding of historical events especially from the perspective of decision making. The data and code for this project are accessible at https://github.com/agiresearch/battleagent and the demo is accessible at https://drive.google.com/file/d/1I5B3KWiYCSSP1uMiPGNmXlTmild-MzRJ/view?usp=sharing .

## introduction an agent is defined as a system that has the ability to perceive its surroundings and make informed decisions based on these perceptions to achieve specific objectives @xcite . recent progress in large language models (llms) @xcite has demonstrated impressive reasoning capabilities @xcite , indicating their potential to serve as the foundation for agents. additionally, the development of large vision language models (vlm) @xcite has facilitated the creation of various agent applications that support multi-modal information interaction @xcite . when combined with external tools, either physical or virtual, these agents employ llm or vlm as their reasoning backbone to determine how tasks should be addressed, how tools should be utilized, and what information should be retained in memory. this enhancement equips agents to manage an array of natural language processing tasks and engage with their environment using language. numerous agent applications have been created using llm and vlm, with a focus on improving reasoning @xcite @xcite , production capabilities @xcite @xcite @xcite , gaming @xcite @xcite , and social simulation @xcite @xcite @xcite , among others. waragent @xcite is the pioneering llm-based mas simulation of historical events, examining the behaviors of systems at the macro level, such as nations and governments, rather than the micro-level simulation of detailed and dynamic events occurring during battles or individual experiences in such dynamic time periods. therefore, battleagent, building on the foundation laid by waragent in historical event simulation, investigates the potential of llm and vlm for detailed historical situation recovery and the exploration of individual experiences within the simulation. to emulate such a complex scenario, our emulation incorporates the following three key features: enhanced 2-d realism features: battleagent emulates detailed interactions within environments, including terrain engagement, temporal progression, and interactions between agents. immersive multi-agent interactions: it integrates mas to facilitate dynamic interactions among agents in battle emulations, accurately reflecting the historical milieu and the intricacies of military engagements, from strategic maneuvers to logistical considerations and communication dynamics. ## emulation setting this section outlines the emulation framework and setting for our research demonstration. we commence with an exposition of the historical context of the four significant european battles that our emulation seeks to emulate: the battle of crécy, the battle of agincourt, the battle of poitiers, and the battle of falkirk. each battle has been selected for its notable use of cold weapons and the strategic bipartite confrontations that characterized warfare during their respective periods. building upon the historical context, we elaborate on the configuration of agents and their designated roles within our emulation framework. ## emulation sandbox in our emulation framework, we concentrate on a relatively straightforward scenario: a bipartite battle. the process begins with (1) setting up the geographical context for the entire scenario, both textual description as well as a visual map, and (2) define the two initial opposing agents, each represents the army of one country. this section will introduce the emulation process: we first present an overview of the sandbox emulation process from a high-level perspective and then delve into the details of the process. this includes how time and location are represented and processed, how agent actions are determined, and how the results of these actions are computed. ## experiment the primary objective of these experiments is to investigate the extent to which agents based on llms and vlms can reasonably emulate historical battles, which are characterized by a high degree of complexity and dynamism. we conduct experiments on 4 distinct historical scenarios, namely the battle of crécy, the battle of agincourt, the battle of falkirk, and the battle of poitiers. the experiments are performed using 3 strong language models and vision-language models: claude-3-opus @xcite , gpt-4-1106-preview @xcite , and gpt-4-vision (openai, 2023). for each scenario and each language model, we execute the emulation 5 times using the same setting to account to randomness, continuing until the casualty figures for both armies converge, or in other words, reach a state of stability. we employ three evaluation metrics as described in ## conclusions and future work in this study, we have demonstrated the potential of llm and vlm to support highly complex and dynamic simulations of historical battles. our emulation sandbox provides a comprehensive evaluation of the emulated battles, including a comparison of casualty figures with historical data and a human analysis of the strategies and tactical maneuvers employed by both armies. we believe that our work can also provide new pedagogical methods for students and researchers interested in historical analysis. by simulating historical battles and presenting the results in an interactive and intuitive way, students can gain a deeper understanding of the complexities and dynamics of warfare.