Large Language Models Play StarCraft II:Benchmarks and A Chain of Summarization Approach

Authors: weiyu  ma
qirui  mi
yongcheng  zeng
xue  yan
yuqiao  wu
runji  lin
haifeng  zhang
jun  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 13:34:22

Abstract: With the continued advancement of Large Language Models (LLMs) Agents in reasoning, planning, and decision-making, benchmarks have become crucial in evaluating these skills. However, there is a notable gap in benchmarks for real-time strategic decision-making. StarCraft II (SC2), with its complex and dynamic nature, serves as an ideal setting for such evaluations. To this end, we have developed TextStarCraft II, a specialized environment for assessing LLMs in real-time strategic scenarios within SC2. Addressing the limitations of traditional Chain of Thought (CoT) methods, we introduce the Chain of Summarization (CoS) method, enhancing LLMs' capabilities in rapid and effective decision-making. Our key experiments included:
1. LLM Evaluation: Tested 10 LLMs in TextStarCraft II, most of them defeating LV5 build-in AI, showcasing effective strategy skills.
2. Commercial Model Knowledge: Evaluated four commercial  models on SC2 knowledge; GPT-4 ranked highest by Grandmaster-level experts.
3. Human-AI Matches: Experimental results showed that fine-tuned LLMs performed on par with Gold-level players in real-time matches, demonstrating comparable strategic abilities.

All code and data from this
study have been made pulicly available at https://github.com/histmeisah/Large-Language-Models-play-StarCraftII

Main paper: 
 introduction: 
 Real-time strategy decision-making and long-term planning are critical AI challenges, necessitating rapid, tactical decisions and strategic adaptability over time. StarCraft II (SC2), one of the world's most popular and challenging e-sports, exemplifies these demands through its dynamic gameplay. Players must manage resources, construct bases, and command armies while making quick decisions and adapting their long-term strategies to evolving battlefield conditions. The game's layered gameplay spans economic management, military strategy, and tactical execution, making SC2 a valuable model for AI research, particularly in reinforcement learning (RL). SC2's complexity, real-time nature, and the fact that it is considered one of the hardest games in the world pose significant challenges for AI systems, requiring them to master various aspects of the game simultaneously. Further details about StarCraft II can be found in the appendix 2. Pioneering efforts, exemplified by Deep-Mind's AlphaStar [24] , have demonstrated significant advancements in this domain, showcasing AI's growing proficiency in strategic gameplay. With the evolution of LLMs in areas like reasoning, planning, and decision-making, these models have begun to show potential in tasks traditionally dominated by RL approaches. Benchmarks such as AGENTBENCH [15] have been instrumental in evaluating these capabilities in multi-turn, open-ended contexts. However, despite these developments, a specific benchmark for assessing LLMs' capabilities in real-time strategy decision-making and long-term planning in environments like StarCraft II is conspicuously absent. Therefore, we have chosen SC2 as the benchmark for evaluating the real-time strategy decisionmaking and long-term planning capabilities of LLMs. Given the lack of language support in existing SC2 environments, we developed TextStarCraft II. Utilizing the python-sc2 framework, TextStar-Craft II converts the complex gameplay dynamics of SC2 into an interactive, text-based format. The python-sc2 interface 1 is utilized to convert game data into text, enabling LLM agents to perform macro-strategic actions through language commands. For micro-strategic actions, we implement a rule-based approach akin to that used by OpenAI Five [2] , employing predefined Python scripts. This allows LLM agents to engage in competition against the game's built-in AI, other LLM agents, and human players through the execution of these scripted actions. Addressing the challenges posed by the intricate decision-making process in SC2, we propose the Chain of Summarization(CoS) method. This approach enhances the capacity of LLM Agents in processing complex information and making strategic decisions by incorporating single-frame and multi-frame summarization modules, each aiding in understanding the immediate game state and processing sequential data for strategy formulation and decision-making.In this study, we conduct a thorough exploration of LLMs' application and effectiveness within SC2 via the TextStarCraft II environment. Our experimental framework includes assessing the CoS method, evaluating the performance of proprietary and fine-tuned open-source LLMs in TextStar-Craft II, testing real-time human-computer interaction, and using StarCraft II-themed questionanswering tasks evaluated by human experts. We also analyze the impact of varying prompts on LLMs, their strategic preferences, and the interpretability of their decision-making processes.Our contributions are manifold:• TextStarCraft II Development: We present TextStarCraft II, a novel environment that not only enables the evaluation of LLMs in strategic gaming contexts but also supports real-time human-computer interactions. • Chain of Summarization Method and Agent: By introducing the CoS method and releasing an open-source agent, we offer a powerful example and a high-level interface for the community, fostering further development and interaction with TextStarCraft II. • Diverse Evaluations of LLMs: Our extensive evaluations encompass testing LLMs against the game's built-in AI, assessing their understanding of StarCraft II through experts reviews, and conducting human-AI matches. These methodologies underscore LLMs' proficiency in strategic decision-making and their potential for human-like gameplay.2 Background: StarCraft IIStarCraft II (SC2) is a real-time strategy (RTS) game developed by Blizzard Entertainment, renowned for its strategic depth and complexity. It features three distinct playable races: Terrans (humans with advanced technology), Protoss (a psionic species with powerful abilities), and Zerg (a biologically advanced, insectoid race), each with unique units, structures, and gameplay mechanics. The game's core elements include resource management, base construction, unit production, and tactical combat. Players must efficiently gather resources (minerals and vespene gas), construct and expand bases, compose balanced armies, and execute strategic plans while adapting to opponents' moves. This multi-faceted gameplay requires players to make rapid decisions under incomplete information, balancing short-term tactics with long-term strategy. SC2's complexity is further enhanced by its asymmetric design, where each race has distinct strengths and weaknesses, leading to a rich variety of strategies and counter-strategies. The game also incorporates elements of economics, technology progression, and territorial control, making it a comprehensive test of strategic thinking and multitasking ability.Since its release in 2010, SC2 has been a cornerstone of competitive gaming, featuring prominently in major e-sports tournaments worldwide. The game's professional scene is characterized by highlevel play, particularly from South Korean competitors, though it maintains a strong international presence. SC2's strategic complexity has made it a subject of extensive study, not only within the gaming community but also in academic fields such as artificial intelligence and cognitive science.Researchers have utilized SC2 as a testbed for developing and evaluating AI algorithms, particularly in areas of hierarchical planning, multi-agent coordination, and decision-making under uncertainty. The game's rich strategic landscape, combined with its clear victory conditions and quantifiable performance metrics, makes it an ideal environment for benchmarking AI systems against human expertise in complex, real-time domains. Moreover, SC2 has been used to study human cognition, particularly in areas of attention allocation, decision-making under time pressure, and skill acquisition. The game's replay system, which records every action taken by players, provides researchers with a wealth of data for analyzing human performance and strategy evolution over time.At 15:28 game time, our situation is as follows: We have 49 workers, 2970 minerals, and 4884 gas. Our supply cap is at 131 with 33 supply left, and our army supply is 41. We have six bases and a decent number of buildings and units. 
current game stage:: 
 Based on the game time and resource availability, we are still in the mid-game stage.Consider building more Gateways and producing units like Stalkers and Immortals to strengthen our ground army. Additionally, expanding our shield battery count can provide better defense against ground attacks.Players choose one of three races to control, each offering a unique array of units and technologies. Games typically involve gathering resources, expanding bases, and commanding armies to defeat enemies. The game's asymmetrical balance is a defining characteristic, with each race having different units, strengths, and weaknesses, requiring distinct strategies. 
related work: 
 StarCraft II Full Game AI: StarCraft AI research, initially focused on StarCraft I with developments like BiCNet [19] for multi-agent coordination, has significantly advanced in the StarCraft II era. The release of PySC2 [23] by DeepMind, coupled with Blizzard's game replays, propelled this research field. A key breakthrough was AlphaStar [24] , which achieved Grandmaster level and defeated top players, demonstrating the potential of RL in complex environments.Subsequent research expanded upon these foundations. Mini-AlphaStar [13] simplified input variables without compromising learning effectiveness. TG [12] and HierNet-SC2 [14] explored efficient RL strategies, with the latter bypassing supervised pre-training. AlphaStar Unplugged [16] represented a leap in offline RL using human replays. TStarBotsX [7] and SCC [26] furthered federated learning approaches, achieving notable success against master and grandmaster level players.Recent advancements include DI-star 2 , which is accessible for home computer deployment, and ROA-Star [9] , enhancing AlphaStar's training framework with goal-conditioned exploiters and refined opponent modeling techniques. ROA-Star's practical tests against professional players have shown impressive results, marking significant progress in real-time strategy AI.LLM Agent and Benchmark: The introduction of GPT3.5 [17] has significantly propelled the research on LLM agents forward. Projects such as React [29] and AutoGPT3 laid the groundwork for more sophisticated implementations. Within the MineDojo environment [5] 4 , initiatives like GITM [33] , Voyager [25] , and others [6, 28, 10] have underscored LLM agents' adaptability to a variety of tasks and expansive open-world scenarios. Additionally, environments like TextWorld [4] and ALFWorld [20] enrich agent training by integrating text-based strategy and action execution across simulated and visual contexts, facilitating advanced generalization and adaptive learning. Additionally, optimization methods such as Reinforcement Learning with Human Feedback (RLHF) [31] have also improved the performance of large language models.Further advancements in multi-agent coordination and virtual social dynamics have been achieved through MetaGPT [8] , Camel [11] , and Generative Agents [18] . Benchmarking platforms such as AGENTBENCH play a critical role in evaluating these developments, with AGENTBENCH examining decision-making in comprehensive, open-ended contexts.In StarCraft II, despite the development of advanced AI agents, there remains a gap in evaluating LLMs, especially in real-time strategy and long-term planning. This led to the creation of TextStar-Craft II, an environment tailored for testing LLMs in these specific aspects, filling a critical need for natural language interaction capabilities in AI research.3.1 Units and Buildings: We have a diverse unit composition, including Voidrays, and a decent number of buildings, including a Cybernetics Core, Twilight Council, and Robotics Facility. We also have a good number of Pylons and gas buildings.3.2 Economy: Our economy is stable, with a sufficient number of workers and a good balance of minerals and gas.3.3 Technology: We have started researching Warpgate and Protoss Air Armor Level 1. However, there is room for further technological development and research.Ensure that our workers are efficiently gathering resources. Consider expanding to new bases to further boost our economy and support our unit production.Terrans: Known for their adaptability and defensive capabilities, Terrans have versatile units and a unique building mechanic (buildings can lift off and move). They are human exiles from Earth. Zerg: The Zerg are all about rapid proliferation and adaptation. They produce large numbers of units quickly and can evolve them into more advanced forms. They rely on overwhelming their opponents with sheer numbers and have the ability to rapidly spread across the map. Protoss: Protoss units and buildings are generally more expensive (in terms of in-game resources) but highly powerful and efficient. They have strong shields, advanced technology, and can warp units directly onto the battlefield.• Building Mobility: Terran structures, unique amongst the races, have the ability to lift off and relocate, allowing for strategic repositioning, surprise tactics, and base migration. This feature is particularly useful for avoiding threats, scouting, and creating forward unit production points. • MULEs and Macro Mechanics: Terran Command Centers can call down MULEs (Mining Unit, Lifting Equipment), temporary units that gather resources at a much higher rate than SCVs, the standard worker unit. This ability provides a significant boost to the Terran economy over short periods, allowing for bursts of increased production. • Bunkers and Siege Warfare: The Terrans have a defensive style of play, with structures like Bunkers and units like the Siege Tank. Bunkers can be garrisoned with infantry units to improve their survivability and increase the Bunker's firepower. Siege Tanks have a transformation ability, allowing them to switch to a stationary siege mode with increased range and damage, ideal for both defense and area control.Each race's unique mechanics require players to adopt very different strategies for each, contributing to the game's dynamic competitive environment. Mastering the utilization of these racial featuresalong with understanding those of potential opponentsis crucial for success in "StarCraft II."Claude2: Here's an overview of the unique racial mechanics for each of the three StarCraft II races:• -Larva -Zerg units are produced from larva generated by Hatcheries. This allows the Zerg player to quickly amass large armies. The Zerg are a swarm-based race that overwhelms their enemies with sheer numbers. They have a variety of cheap and expendable units, such as Zerglings and Roaches, as well as more powerful units such as Hydralisks and Ultralisks. Zerg players are also known for their ability to quickly expand their bases and produce units.Here are some of the unique racial mechanics of the Zerg:• Inject Larva: Zerg Hatcheries produce Larva, which can then be morphed into different types of units. The Inject Larva ability allows Zerg players to increase the production rate of their Hatcheries. • Creep Spread: Zerg Creep is a substance that spreads from Zerg structures and units. Zerg units gain movement speed and attack speed on Creep. Zerg players can use Creep to quickly move their armies around the map and to gain an advantage in battles. • Overlord Speed: Overlords are Zerg flying units that provide vision and can transport units.Zerg players can upgrade the speed of their Overlords, which allows them to quickly scout the map and to transport their armies to different locations. • Zerg Rush: A Zerg Rush is a strategy where Zerg players produce a large number of cheap and expendable units early in the game and attack their opponent. Zerg Rushes can be very effective, but they can also be countered by players who are prepared for them.The Protoss are a highly advanced race that relies on powerful technology and psionic abilities. They have a limited number of units to choose from, but each unit is very powerful. Protoss players are known for their strong economies and their ability to build powerful armies.Here are some of the unique racial mechanics of the Protoss:• Warp Gate: Warp Gates are Protoss structures that allow Protoss players to warp in units directly from their Nexus. This allows Protoss players to quickly produce units at any location on the map.• Chrono Boost: Chrono Boost is a Protoss ability that can be used to speed up the production of units or the research of technologies. Protoss players can use Chrono Boost to gain an advantage in the early game or to quickly produce a powerful army.• Shields: Protoss units have shields, which are a layer of energy that protects them from damage. Shields regenerate over time, but they can be depleted by enemy attacks. Protoss players can use their shields to their advantage by engaging in hit-and-run tactics or by using their shields to absorb damage while their units attack.• Psionic Storm: Psionic Storm is a Protoss spell that creates a storm of psionic energy that damages enemy units in a large area. Protoss players can use Psionic Storm to clear out enemy armies or to damage enemy structures.The Terrans are a versatile race that combines human ingenuity with advanced technology. They have a wide variety of units to choose from, including Marines, Siege Tanks, and Battlecruisers. Terrans are also known for their powerful mobile structures, such as Command Centers and Factories.Here are some of the unique racial mechanics of the Terrans:• MULE: MULEs are Terran worker units that can be called down from Orbital Command Centers. M.U.L.E.s mine minerals at a faster rate than SCVs, which can help Terran players to quickly build up their economies.• Supply Drops: Terran Supply Depots provide supply, which is needed to produce units. Terran players can also call down Supply Drops from their Command Centers, which provides additional supply. This allows Terran players to quickly build up their armies.• Siege Tanks: Siege Tanks are Terran tank units that can transform into a siege mode. In siege mode, Siege Tanks have a longer range and deal more damage. Terran players can use Siege Tanks to defend their bases or to attack enemy bases from a distance.• Medivacs: Medivacs are Terran flying units that can transport units and heal them. Terran players can use Medivacs to quickly move their armies around the map and to keep their units alive in battle.Each race in StarCraft II has its own unique strengths and weaknesses. Zerg players are known for their ability to quickly produce large armies, Protoss players are known for their strong economies and powerful units, and Terran players are known for their versatility and mobile structures.The best way to learn about the unique racial mechanics of each race is to experiment with them and to play against other players. • Bio-Mech Composition: Terran often uses a combination of infantry (Marines, Marauders) and mechanical units (Siege Tanks, Hellbats) against Zerg. This composition allows for mobility and firepower.• Harassment: Terran players often employ drop tactics, using Medivacs to drop units into Zerg bases to disrupt their economy and force them to split their forces.• Widow Mines: Widow Mines can be used to deter Zergling and Baneling attacks, and they can also be dropped into Zerg mineral lines to deal economic damage.GPT4: A common strategy involves harassment to prevent the Zerg from comfortably expanding and massing their forces. Terrans might use Hellions to attack Zerg's worker units or dropships to move units behind enemy lines. The overall aim is to control the pace, forcing Zerg to react to Terran threats while building a strong economy and army for a decisive push.Claude2: Terran often opens with a Reaper expand into Hellions for map control and harassment. This transitions to Marine/Medivac for mobility and bio strength. Tanks added for splash against Zerglings.Bard: A common strategy for Terran vs. Zerg is to go for a macro build, such as Bio (Marines, Medivacs, and other support units) or Mech (Siege Tanks, Hellions, and other mechanical units).Terran players can also try to rush their Zerg opponent with early aggression, such as a Reaper Rush or a Marine/Marauder Rush.Q2: TvT: What's a typical strategy in Terran vs. Terran matchups?GPT3.5:• Tank Warfare: TvT often involves tank battles. Players set up Siege Tanks defensively and try to gain positional advantages while avoiding being caught unsieged. • Viking Control: Air control is crucial in TvT. Vikings are used to gain air superiority and vision for Siege Tanks. Raven usage for interference matrix can also be a key factor. • Bio vs. Mech: Terran players can choose between a bio-heavy composition (Marines, Marauders) and a mech-heavy composition (Siege Tanks, Hellions). The choice can depend on personal preference and map.GPT4: TvT often revolves around positioning and tank warfare. Siege Tanks are critical for area control, and players jockey for position and map control. Air superiority via Vikings is also crucial to provide vision and control. Since all units are available to both sides, scouting, deception, and adaptability are key.Both players open Reaper expand for scouting information. Focus is on securing additional bases for economy. Tanks, Vikings and air control are critical for positioning and engagements.Bard: A typical strategy in Terran vs. Terran matchups is to go for a macro build, such as Bio or Mech. Terran players can also try to scout their opponent's build early and counter it with a specific composition. For example, if a Terran player sees that their opponent is going for Bio, they can counter with Siege Tanks and Hellions.Q3: TvP: How do Terran players generally approach a Protoss opponent? GPT3.5:• Bio Play: Terran often employs bio compositions (Marines, Marauders, Medivacs) against Protoss. Micro-intensive battles and mobility are key. • Ghost Usage: Ghosts can snipe high-value Protoss units like Templars and Observers. They also have EMP to drain Protoss shields and energy. • Liberator Zones: Liberators can establish zones of control and disrupt Protoss positioning with their anti-ground mode.GPT4: Terrans often use a mixture of bio units (Marines and Marauders) supplemented with Medivacs for healing and mobility against Protoss. Harassment with drops to disrupt the Protoss economy and tech is common. Terrans need to be aggressive, as a fully teched and maxed Protoss army can be difficult to defeat head-on.Claude2: Terran applies early pressure with units like Reapers, Hellions and Widow Mines. Goal is to slow Protoss economy and tech while expanding themselves. Marines and Ghosts counter late-game Protoss power units.Bard: Terran players generally approach a Protoss opponent by trying to expand their economy and build up a strong army. Terran players can also try to rush their Protoss opponent with early aggression, such as a Reaper Rush or a Hellion Rush.GPT4: Zerg often faces bio-ball compositions (Marines, Marauders, Medivacs) or mechanical units (like Tanks and Thors). Zerg needs to expand quickly, defend against harassment, and may use Banelings, Zerglings, and Hydralisks for defense transitioning to more robust units like Ultralisks or Brood Lords depending on Terran composition.Claude2: Ling/Bane for Bio defense. Fast spire into Mutalisk harass. Transition to Ultralisks supported by Queens and Infestors for the often mech-heavy Terran end-game.Bard: Zerg players counter Terran tactics by using their superior mobility and economy to their advantage. Zerg players can also use their powerful units, such as Ultralisks and Brood Lords, to defeat Terran armies Q7: PvZ: How should Protoss approach a Zerg opponent?GPT3.5:• Gateway Expand: Protoss players often open with a Gateway expansion, using units like Adepts and Stalkers to control the early game.• Tech Choices: Protoss can opt for Stargate play (Phoenixes or Oracles) for harassment, or they may go for a Robo facility for Immortals or Disruptors.• High Templars: Templars are used for their Psionic Storm ability to deal with large Zerg armies.GPT4: Protoss players must secure a solid economic base while defending against Zerg's potential for rushes and harassment. Walling off chokepoints with buildings and using units like Zealots and Sentries to defend is common early game. They rely on powerful AoE units (like Colossi and High Templars) later in the game, often leveraging tech advantages for strong pushes.Claude2: Fast expand into Immortal/Archon/Chargelot to defend Zerg attacks. High Templar for storms against Hydras. Skytoss deathball critical for late game.Bard: Protoss players approach a Zerg opponent by trying to expand their economy and build up a strong army of powerful units, such as Zealots, Stalkers, and Immortals. Protoss players can also use their Colossi to deal damage to Zerg units from a distance.Q8: PvP: What's the strategy for Protoss vs. Protoss games? GPT3.5:• Stalker-Sentry: Early PvP often features Stalkers and Sentries for map control and defense.• Disruptor Play: Disruptors are a key unit in PvP, used for their ability to deal massive area damage to enemy units.• Blink Stalker Micro: Micro-management of Blink Stalkers can be a deciding factor in PvP engagements.GPT4: Early game revolves around preventing enemy harassment and scouting to anticipate tech choices. The mid-game often sees a lot of positioning battles and potential for blink Stalker harassment. Since Protoss units are powerful but costly, efficient trades and protecting key units are paramount, with late-game often seeing advanced units like Carriers or Tempests.Claude2: Robo for Immortals or fast expand with Blink Stalkers. Disruptors and High Templar for splash against massed Gateway units. Whoever gets Tempests wins the late game.The strategy for Protoss vs. Protoss games typically involves trying to scout their opponent's build early and counter it with a specific composition. For example, if a Protoss player sees that their opponent is going for a Colossi build, they can counter with Immortals and Void Rays. Strategy for Protoss:• Force Fields: Use Sentries to create Force Fields to segment the Roach-Hydra army, preventing it from engaging as a whole.• Colossi: Colossus units deal area damage and are effective against clumps of Roaches and Hydralisks.• Immortals: Immortals are strong against Roaches due to their bonus damage against armored units.• Archons: Archons have splash damage and can help clear out Hydralisks.Counterplay for Zerg:• Tech to Hive: Transition to Hive tech for units like Vipers, which can abduct key enemy units or use Blinding Cloud to reduce the effectiveness of enemy ranged units.• Hydra Speed Upgrade: Research Muscular Augments for Hydralisks to improve their mobility.• Burrow: Use Burrow for Roaches to save them from being picked off and to regain health.• Drop Play: Drop Roaches or Hydralisks into the Protoss main base for harassment to divert their attention. Creep Spread: Maintain good Creep spread to ensure mobility and vision.GPT4: When Protoss players face a Zerg opponent focusing on Roach-Hydra, they're dealing with a versatile force that can be potent in a head-on fight due to its range, damage, and health, particularly in the mid-game.• Strategic Positioning: Force engagements in narrow passages where the Zerg cannot leverage their numbers. Simultaneously, avoid open areas where the Zerg army can fully surround your forces.• High Templars: These units are crucial. Their Psionic Storm ability deals massive damage to biological units, which makes them particularly effective against Hydralisks and clumped-up Roaches.• Colossi: With their extended thermal lance upgrade, they can exploit their superior range and area-of-effect damage to decimate tightly packed Roach-Hydra armies.• Sentries: Force Fields can cut off portions of the Zerg army, preventing them from engaging all at once, or Guardian Shield can reduce incoming ranged damage.• Harassment: Divide the Zerg's attention. Warp Prisms, Zealot run-bys, and Dark Templars can force the Zerg player to split their army, making it less effective in a direct engagement.Claude2: Against Roach-Hydra, go Immortal/Archon with chargelots. Immortals counter Roaches, Archons counter Hydras. Focus fire with Immortals.alternatively, fast tech to Colossus. Use Sentries for forcefields to split Zerg army. Expand behind and get High Templar for storms.Bard: Effective Strategies:• Go for a Colossus build. Colossi are very effective against Roaches and Hydras, and they can also deal damage to Zerg structures.• Use Void Rays to pick off Roaches and Hydras from a distance. Void Rays are also very effective against Zerg Overlords, which can help you to deny your opponent vision and to make it more difficult for them to attack you.• Use Warp Gates to quickly produce units and to respond to Zerg attacks. Warp Gates allow you to warp in units directly from your Nexus, which can help you to quickly build up your army and to defend your base. Effective Counterplays:• Use Observers to scout your opponent's base and to see what kind of army they are building. This will give you time to prepare for their attack and to build the appropriate counter-units.• Use Siege Tanks to deal damage to Mutalisks and Zerglings from a distance. Siege Tanks are also very effective against Zerg Overlords, which can help you to deny your opponent vision and to make it more difficult for them to attack you.• Use Medivacs to heal your units and to transport them around the map. Medivacs can also be used to drop your units into the middle of a Zerg army, which can be a very effective way to engage your opponent. Effective Counterplays:• Use Missile Turrets and Bunkers to defend your base from Mutalisk attacks. Missile Turrets and Bunkers are both very effective against Mutalisks, and they can also be used to defend your workers and your structures.• Use Ravens to detect Mutalisks and to cast Interference Matrix on them. This will make it more difficult for your opponent to control their Mutalisks and to attack you.• Use Vikings to shoot down Mutalisks. Vikings are the only Terran unit that can attack air units without moving, which makes them very effective against Mutalisks.Q3: TvP: When faced with an archon-zealot push, what are the best strategies and counters for Terran?GPT3.5: TvP: Against Archon-Zealot Push:Strategy for Terran:• Kite and Focus Fire: Kite the Zealots with your ranged units (e.g., Marines and Marauders) and focus fire on Archons.• Medivac Micro: Use Medivacs to pick up and reposition units to avoid the melee engagements.• Widow Mines: Position Widow Mines to hit the Zealots and soften the Archons.• Ghosts: EMP from Ghosts can drain the shields of Protoss units, including Archons.Counterplay for Protoss:• Force Fields: Create Force Fields to segment the Terran army and prevent kiting.• Immortals: Immortals are strong against Archons due to their bonus damage against armored units.• High Templars: Use Psi Storm to deal AoE damage to the Terran infantry.• Air Units: Consider incorporating air units like Void Rays or Phoenixes for added versatility.• Guardian Shield: Use the Guardian Shield ability from Sentries to reduce incoming ranged damage.Remember that these strategies and counterplays are not exhaustive, and adaptability is key in Star-Craft II. The specific outcome of battles can depend on micro-management, unit composition, upgrades, and positioning, so always be prepared to adjust your tactics based on the evolving situation in the game.GPT4: Archon-Zealot (often with some Immortals) is a powerful, close-quarters Protoss army composition that relies on brute force and strong front-line units.• Ghost Usage: Ghosts' EMP Round is highly effective in draining the shields of Archons, Zealots, and Immortals, significantly weakening their upfront staying power in battles.• Liberators and Siege Tanks: The use of Liberators with Defender Mode and Siege Tanks can create a zone of control that Zealots and Archons cannot approach easily. Kiting into these zones can decimate a Protoss army that's chasing.• Micro-intensive Kiting: Using bio forces (Marines and Marauders) supplemented with Medivacs for healing, a player can continuously kite against Zealots and Archons, which lack ranged attacks.• Sim City: Terrans can build in a way that chokes off paths (using Supply Depots, Bunkers, etc.), restricting the mobility and attack angles of melee units like Zealots and forcing Archons to funnel through tight spaces where they can't leverage their splash damage.Each of these scenarios emphasizes not just the composition of your forces but also the importance of control, positioning, adaptability, and game sense. "StarCraft II" is a game of constant decisionmaking, and successful players must be able to read their opponents and adjust their strategies on the fly.Claude2: Marines and Marauders counter Zealots, while Ghosts and Liberators counter Archons. Kite with bio and emp Archons. Siege Tanks in tank mode punish Zealot chargelots. Well-spread mines damage clumped Zealots. Scan aggressively to preemptively snipe Observers and enable Widow Mines.Bard: Effective Strategies:• Go for a Bio build. Bio units are very effective against Zealots and Archons, and they can also deal damage to Protoss structures. • Use Siege Tanks to deal damage to Zealots and Archons from a distance. Siege Tanks are also very effective against Protoss Colossus, which can help you to defend your base and to make it more difficult for your opponent to attack you. • Use Medivacs to heal your units and to transport them around the map. Medivacs can also be used to drop your units into the middle of a Protoss army, which can be a very effective way to engage your opponent.Effective Counterplays:• Use Missile Turrets and Bunkers to defend your base from Archon and Zealot attacks. Missile Turrets and Bunkers are both very effective against Zealots and Archons, and they can also be used to defend your workers and your structures. • Use Scans to detect Protoss units and to see what kind of army they are building. This will give you time to prepare for their attack and to build the appropriate counter-units. • Use Snipers to pick off Protoss units from a distance. Snipers are very effective against Zealots and Archons, and they can also be used to deal damage to Protoss Colossus.https://github.com/BurnySc2/python-sc2https://github.com/opendilab/DI-starhttps://github.com/Significant-Gravitas/AutoGPThttps://github.com/MineDojo/MineDojo 
textstarcraft ii: 
 TextStarCraft II provides a text-based interface for LLMs within the SC2 environment, utilizing the python-sc2 framework to translate complex gameplay into text. Key components include the Observation-to-Text Adapter and the Text-to-Action Adapter.TextStarCraft II stands out from other text-based environments like TextWorld and ALFWorld due to its more complex and dynamic gameplay. It requires agents to manage multiple aspects such as resource allocation, base building, and military strategy in real-time, challenging both their adaptability and decision-making. Additionally, the environment demands advanced language understanding to interpret and execute more open-ended and diverse commands, enhancing the need for sophisticated natural language processing capabilities. We will introduce the main components of TextStarCraft II below.Observation TextStarCraft II's observation space is designed to equip LLM agents with essential game insights, effectively navigating the fog of war in StarCraft II. The observations encompass six key categories:• Resources: Game resources and supply levels. ("Minerals: 500, Vespene Gas:200 ")• Units: Types and quantities of the players units.(e.g., "15 Zealots, 5 Carriers")• Buildings: Information on the players buildings.(e.g., "2 Gateway, 1 Nexus")• In-Process Activities: Ongoing construction and production data.(e.g., "Researching Warpgate: 70• Enemy Status: Visible enemy units and buildings• Research Progress: Updates on the player's technological advancements.This structured approach in the observation space enables LLM agents to efficiently process and utilize vital game data for strategic decision-making in TextStarCraft II.Action There are mainly two types of actions: Macro Actions and Micro Actions. -Building construction (e.g., "Build Cybernetics Core") -Unit production (e.g., "Train 5 Probes (Protoss worker)") -Research (e.g., "Research Blink Tech") -Tactical maneuvers (e.g., "Scout with 1 probe", "Attack enemy base")• Micro Actions: Script-managed for precise placements and targeting, not directly controlled by the agent.The reward function R is crucial for aligning agent behavior with the game's objectives, assigning values of {-1, 0, 1} based on losing, drawing, or winning a match. Our strategy seems to be focused on air dominance with Voidrays. We should continue to leverage our air superiority while expanding our technological capabilities.Continue researching upgrades for our air units, such as Protoss Air Armor Level 2 and Level 3. Additionally, consider researching upgrades for our ground units, such as Protoss Ground Weapons and Armor. In Section 6.3, we introduced the concept of policy interpretability within our LLM agent. We now present four illustrative examples to demonstrate how the LLM agent can generate reasonable strategies and decisions in TextStarCraft II. Each example comprises three components:• L1 Results Figure : This displays the input of the multi-frame summarization method.• LLM Output: The output of the multi-frame summarization method, which includes the LLM's analysis, suggestions, and decisions.• Decision Figure : Illustrates the actual implementation within the StarCraft II game engine. • 1. Game Overview: At 00:46 game time, our current situation is as follows. We have 15 workers and 105 minerals. Our supply cap is at 23 with 7 supply left, and our army supply is 16. We have one base, one pylon, and two gas buildings. There are no known enemy units.• 2. Current Game Stage: Based on the limited resources and small army size, we are still in the early game stage.• 3. Our Situation: -3.1 Units and Buildings: We have a small worker force and limited infrastructure. We have one base, one pylon, and two gas buildings. There are no units present except for the workers. -3.2 Economy: Our economy is just starting, with a small number of workers and limited resources. -3.3 Technology: There is no evidence of any technological development or research at this stage.• 4. Our Strategy: At this early stage, our strategy should focus on expanding our economy, establishing more bases, and increasing our worker count. We should also consider scouting to gather information about the enemy's strategy.• 5. Enemy's Strategy: Since there is no information about enemy units, it is difficult to determine their strategy. However, it is important to remain vigilant and be prepared for any potential attacks. • 6. Key Information: The most important aspect at this moment is to quickly expand our economy and increase our worker count. This will provide us with a stronger economic foundation and allow us to produce more units and buildings.Suggestions:• 1. Our Strategy: Prioritize expanding our economy by building additional bases and increasing our worker count. Consider scouting to gather information about the enemy's strategy.• 2. Units and Buildings: Focus on building additional pylons to increase our supply cap and support future unit production. Consider constructing additional structures such as gateways or a robotics facility to start producing units. • 3. Economy: Allocate workers to gather minerals and gas efficiently. Aim to saturate our current base and expand to new resource locations as soon as possible. • 4. Technology: At this early stage, it is not necessary to focus on technology research.However, consider building a cybernetics core to unlock more advanced units and upgrades in the future. • Decisions:-0: <BUILD NEXUS> -1: <BUILD PYLON> -2: <BUILD GATEWAY> -3: <TRAIN PROBE> -4: <SCOUTING PROBE>The Protoss are currently employing Probes for scouting purposes. • 1. Game Overview: At 04:25 game time, our situation is quite challenging. We have limited resources with 215 minerals and 128 gas. Our supply cap is at 54 with 16 supply left, and our army supply is just 8. We currently have two bases.• 2. Current Game Stage: Based on the limited resources and small army size, we are in the early game stage.• 3. Our Situation:-3.1 Units and Buildings: We have a modest army consisting of 4 Zealots. We are in the process of constructing additional Gateways, a Robotics Facility, and a Stargate. -3.2 Economy: Our economy is struggling with minimal resources. We need to expand our resource collection urgently. -3.3 Technology: We have already researched Warp Gate technology, which is a positive development. However, we lack diversity in our units and are still in the process of building advanced tech structures.• 4. Our Strategy: It appears we are gearing up for a tech-heavy approach with the construction of advanced tech structures. However, our current economic state is not supporting this strategy well.• 5. Enemy's Strategy: The enemy seems to be focusing on expanding with Hatchery and Extractor but has also started building a Spawning Pool, indicating the potential for Zerg unit production.• 6. Key Information: The most crucial aspect at this moment is our resource scarcity. We need to address this issue immediately to support our tech-heavy strategy and build a more substantial army to defend against potential enemy threats.Suggestions:• 1. Our Strategy: Given our low resources, it's crucial to prioritize resource expansion in the short term. Build additional Probes to maximize mineral and gas collection. Consider a defensive strategy while we bolster our economy.• 2. Units and Buildings: Continue constructing the planned tech structures, but also prioritize the construction of additional Gateways for faster unit production. Warp-in Zealots as a cost-effective defensive measure. • 3. Economy: Expand to new resource locations to increase income. Keep worker production consistent and utilize Chrono Boost on Nexus to accelerate Probe production. • 1. Game Overview: At 13:28 game time, our situation is as follows: We have 54 workers, 1135 minerals, and 975 gas. Our supply cap is at 155 with 7 supply left, and our army supply is 87. We have six bases.• 2. Current Game Stage: Based on the game time and resource availability, we are in the mid to late-game stage.• 3. Our Situation: -3.1 Units and Buildings: We have a decent worker count and a strong army composition with Stalkers, Colossi, Disruptors, Observers, Phoenixes, and Void Rays. Our building count is also significant, with multiple pylons, gateways, robotics facilities, and a robotics bay. -3.2 Economy: Our economy is stable, with a good balance of minerals and gas. We have enough workers to maintain our resource collection efficiently. -3.3 Technology: We have already researched Warpgate technology, which is essential for unit production. However, there is no information about further technological advancements or upgrades.• 4. Our Strategy: Our strategy seems to be focused on a strong ground and air army composition, utilizing Stalkers, Colossi, Disruptors, Phoenixes, and Void Rays. We should continue to leverage our army strength and maintain a strong economy.• 5. Enemy's Strategy: The enemy has a diverse unit composition, including Broodlords, Overlord Transports, Corruptors, Overseers, and ground units. They seem to be focusing on air superiority and potentially using Broodlords for high ground control.• 6. Key Information: The most crucial aspect at this moment is our strong army composition, which includes anti-air units like Phoenixes and Void Rays. However, we should be cautious of the enemy's Broodlords, which can pose a significant threat to our ground forces. Additionally, we need to focus on further technological advancements and upgrades to stay ahead in the game. Suggestions:-1. Our Strategy: We should continue to strengthen our army composition by adding more anti-air units like Phoenixes and Void Rays. Additionally, we should consider countering the enemy's Broodlords with appropriate units or tactics. -2. Units and Buildings: Given the enemy's air dominance, it would be beneficial to increase the production of Phoenixes and Void Rays. Additionally, consider building more Stargates to support the production of these units. It may also be necessary to reinforce our ground forces with additional Stalkers and Colossi. In subsection 6.2, we documented the responses from various LLMs, encompassing GPT3.5, GPT4, Claude2, and Bard, spanning five distinct question categories. 3) "Legacy of the Void" (released in 2015) -The final installment concentrates on the Protoss and features the character Artanis as he works to unite the Protoss and confront a cosmic threat known as the "Dark Voice" or "Amon."The game is recognized for its competitive multiplayer mode, where players from around the world compete in leagues. Each of the three races in the game (Terrans, Zerg, and Protoss) has its unique set of units and structures, which results in very different strategies and requires a deep understanding of the game to master."StarCraft II" has also been a significant part of the professional e-sports scene, with numerous tournaments worldwide, including the prestigious StarCraft II World Championship Series (WCS). The game's emphasis on quick decision-making, strategic diversity, and real-time tactical execution contribute to its standing as a respected e-sport. -Players can play as three factions or races -the Terrans (humans), the insect-like Zerg, and the technologically advanced Protoss.-The gameplay involves gathering resources, building bases and armies, and battling opponents in real-time. The goal is to destroy the enemy base.-StarCraft 2 has a thriving eSports scene with many professional tournaments and leagues. Top players can earn significant prize money.-It's considered one of the greatest and most influential real-time strategy franchises to date, known for its balanced asynchronous gameplay, strategic depth, and cinematic storytelling.-Blizzard continues to support StarCraft 2 with updates, new co-op commanders, and balance changes to keep the multiplayer aspect fresh. Game Overview:• Release: StarCraft II was released in three separate episodes. The first episode, "Wings of Liberty," was released in July 2010. The second episode, "Heart of the Swarm," followed in March 2013, and the final episode, "Legacy of the Void," was released in November 2015. • Genre: StarCraft II is a real-time strategy (RTS) game that focuses on resource management, base building, and strategic combat. • Developer and Publisher: Developed and published by Blizzard Entertainment.• Setting: The game is set in a futuristic science fiction universe and features three distinct races: Terrans, Zerg, and Protoss. Each race has its own unique units, buildings, and gameplay mechanics. Single Player: Each episode of StarCraft II has a single-player campaign that follows the story of one of the three races. The campaigns offer engaging storytelling and missions with varying objectives.Races:• Terrans: Terrans are humans who have been exiled from Earth and are known for their adaptability and use of advanced technology. They have a wide range of mechanical units and structures. • Zerg: Zerg are a highly adaptable and insectoid alien race that can rapidly grow and overwhelm their opponents with numbers. They are known for their biological units and ability to spread creep. • Protoss: Protoss are a technologically advanced alien race with powerful psionic abilities.They have advanced units and structures, and their gameplay is focused on precision and strategy.Esports Scene:StarCraft II has a thriving esports scene, with numerous tournaments and professional players from around the world.Major tournaments include the World Championship Series (WCS), Intel Extreme Masters (IEM), GSL (Global StarCraft II League), and BlizzCon.Professional players and teams compete for significant prize money and recognition in these tournaments.The game's competitive nature and high skill ceiling make it a popular choice for both casual and professional players.StarCraft II is the successor to the original StarCraft, which is considered one of the most influential and iconic real-time strategy games of all time.StarCraft II continued the legacy of its predecessor and expanded the game's lore and gameplay mechanics.The game has a dedicated fanbase and has had a significant impact on the esports industry.StarCraft II has a vibrant modding and custom content community. Players can create and share custom maps, game modes, and mods through the StarCraft II Arcade.This community-driven aspect adds to the game's longevity and diversity of gameplay experiences.While my knowledge is based on information available up until September 2021, StarCraft II has likely continued to evolve and develop in the years since. If you have specific questions or need more recent information about the game or its esports scene, I recommend checking official sources and community forums for the latest updates.GPT4: "StarCraft II" is a monumental figure in the world of real-time strategy (RTS) games and e-sports. Developed by Blizzard Entertainment, it's known for its demanding skill level, strategic diversity, and an active professional scene. Here's a comprehensive overview:1. Overview:Released in 2010, "StarCraft II" is set in a science fiction universe in the 26th century and focuses on the interstellar struggles between three species: the Terrans (humans exiled from Earth), the Zerg (a super-species of assimilated life forms), and the Protoss (a technologically advanced species with psionic abilities)."StarCraft II" was released as a trilogy, with each installment focusing on a different race's perspective in the ongoing interstellar war. "Wings of Liberty": Focuses on the Terran campaign. "Heart of the Swarm": Revolves around the Zerg campaign. "Legacy of the Void": Concludes with the Protoss campaign. 
chain of summarization: 
 The Chain of Summarization (CoS) method, integral to the TextStarCraft II framework, draws inspiration from computer hardware's cache mechanisms and RL's frame skipping techniques. Serving as an enhancement to traditional Chains of Thought (CoT) [27] and as a standard plug-in, CoS refines strategic decision-making in StarCraft II through:• Information Compression: It focuses on key data, reducing overload and sharpening strategic clarity. • Inference Acceleration: This approach speeds up decision-making by providing a more comprehensive view of the game's state.• Global Understanding: CoS equips LLMs with a deeper grasp of game strategies, leading to expert-level decisions.As a versatile tool within the TextStarCraft II framework, CoS can operate both as a standalone plugin, enhancing the environment's utility, and as a direct interface for user interaction with TextStar-Craft II. This dual functionality not only showcases CoS as an exemplary model for engaging with our environment but also invites further development and customization by the community, broadening the scope of strategic AI research in gaming. CoS includes Single-Frame Summarization, Multi-Frame Summarization, and Action Extraction for the Action Queue.Single-Frame Summarization To make TextStarCraft's raw observation data more comprehensible for LLMs, Single-frame Summarization compresses and extracts key information. This process, denoted as S SF (•), transforms dense TextStarCraft II observations o into a condensed form ô, described by:EQUATIONThere are two approaches to this compression: a language model-based approach using few-shot learning for better alignment with game rules and a faster, rule-based approach for extraction and filtering. In our experiments, the rule-based approach is primarily used for quicker interactions.Multi-Frame Summarization Traditional methods query LLMs at each time step for decisionmaking ( [3] , [32] ). However, this is inefficient for long-duration games like StarCraft II due to high computation costs and slower LLM inference. Our Multi-Frame Summarization method, inspired by caching in computer hardware and frame skipping in RL, addresses these issues. It synchronizes the quick pace of the game with LLM processing, ensuring real-time decision-making efficiency and improved comprehension in complex scenarios. Instead of constant LLM querying, we aggregate for raw observation o in Q obs do 8:Perform Single-Frame Summarization ô = S SF (o)Add ô to the Single-Frame Summarization Queue Q SFS 10:end for11: Perform Multi-Frame summarization σ = S MF (Q SFS ) 12:Apply Chain of Thought reasoning υ = CoT (σ)13: extract K actions (a 1 , a 2 • • • , a K ) = Ex(υ) ▷ CoS end 14:Add the K actions (a 1 , a Add the raw observation o t+1 to the raw observation queue Q obsR ← R + r t 20: end while 21: return total reward R condensed observation information ô for K steps into a period summary σ, described by:EQUATIONThis method enables comprehensive analysis and strategic planning through a series of steps, including situation overview, analysis, strategic planning, opponent strategy analysis, suggestion formulation, and decision-making. This process is formalized as υ, which is the output of CoT reasoning for summarization σ, given by: υ = CoT (σ).(EQUATIONAction Extraction for Action Queue The action queue forms a critical link between the Multi-Frame Summarization results, υ, and the TextStarCraft II environment, facilitating communication between the LLM and the game. Within υ, key components include analysis, suggestions, and decisions. To convert these into actionable steps, we employ regular expression matching and similarity searching in our action extractor, donated as Ex(•). This process populates the action queue with actions ready for execution in TextStarCraft II. From the output υ of CoT reasoning, we utilize the action extractor to extract K actions, as formalized in:EQUATION)The CoS method optimizes decision-making in TextStarCraft II through a streamlined four-stage process: Initially, it sets the initial parameters and transforms the first game frame into text for subsequent analysis. Next, it distills key game observations to provide a concise snapshot of the current situation. Following this, the method translates these summaries into strategic action plans. Finally, it implements the planned actions within the game, thereby completing the decision cycle. This process is depicted in Figure 1 . This approach, which updates actions every few frames, effectively manages the fast-paced dynamics of StarCraft II, thereby proving essential for real-time strategic gameplay. The pseudocode is as shown in Algorithm 1.The enemy seems to be focused on a ground-based army composition, with Roaches, Zerglings, and Hydralisks. They also have some support units like Overseers and Infestors. We should be cautious of potential ground attacks and plan our defenses accordingly.Shortly after its release, "StarCraft II" exploded into the e-sports scene. It built on the legacy of the original "StarCraft" game, which was one of the first globally recognized e-sports. The game is celebrated for its incredibly high skill ceiling and strategic depth, making professional matches fascinating for spectators. The "StarCraft II World Championship Series (WCS)" was the premier competitive circuit for professional "StarCraft II" players, featuring several regional events leading up to a global year-end championship. South Korea, known for pioneering e-sports, has been particularly prominent in the "StarCraft II" scene, producing numerous professional players who dominate in international competitions.6. Community and Legacy:"StarCraft II" has a strong community of players, fans, and dedicated content creators. The game's custom map editor has led to a variety of player-created content, adding to the game's longevity. In October 2020, Blizzard announced the end of active development of new content for "StarCraft II," shifting to a maintenance mode for the game with a focus on balance and ecosystem health. Despite the reduction in developer-driven content, the community and e-sports scene remain active, with third-party organizers continuing to hold tournaments. 
experiment: 
 In our experiment, we detail the setup and key metrics (evaluation metrics detailed in Appendix A.2.) to evaluate macro-strategic decision-making in StarCraft II. We assess the Chain of Summarization's impact on LLM gameplay, compare various LLMs' performance, and evaluate their grasp of Star-Craft II strategies. Our experiments also concludes with human-AI interaction tests. In this section, we assess the performance of closed-source LLMs [21] , Llama2 70B [22] , and fine-tuned open-source LLMs such as ChatGLM3 6B [30] and Qwen 1.8B [1] in the TextStarCraft II environment against the built-in AI at level 5. The experimental results are shown in Table 1 , with the evaluation metrics detailed in Appendix A.2. We tested closed-source LLMs and the unfine-tuned Llama2 70B using the standard CoS method. The closed-source models performed well, while Llama2 70B could not understand the task requirements and was unable to generate commands based on the given prompts.Additionally, we fine-tuned open-source models using the entire dataset of GPT3.5-turbo-16k interaction logs with TextStarCraft II. Due to computational resource limitations, we removed the CoT component, keeping only the inputs and outputs. The results showed that all evaluated closed-source LLMs were capable of defeating the level 5 built-in AI. The fine-tuned open-source models, despite a loss in strategic diversity, still managed to overcome the level 5 AI, predominantly adopting a strategy focused on mass-producing stalkers. spite a reasonable grasp of basic game dynamics, these models faced challenges with more complex elements like the tech tree and supply constraints.Evaluation Methodology: To gauge the depth of LLMs' StarCraft II knowledge, we tested models, including ChatGPT (GPT3.5), GPT-4, Claude2, and Bard, across five areas: Basic Knowledge, Racial Mechanics, Typical Strategies, Standard Build Orders, and Classic Strategies and Counterplays (detailed in Appendix G). We used a double-blind evaluation, with both Grandmaster-level human experts and GPT-4 assessing the responses to ensure unbiased scoring.Evaluation Results: Both human experts and GPT-4 assessed the LLMs, illustrated in Figures 2 and Table 6 , leading to the following insights: In the evaluation of LLMs on a set of complex questions, GPT-4 and ChatGPT demonstrated the best performance, with GPT-4 receiving high scores from both itself and human experts. Claude2 had mixed results, with GPT-4 rating it higher than human experts did. Bard struggled with the complex questions, receiving the lowest scores, especially on questions 3, 4, and 5. Overall, the LLMs were ranked as follows: GPT-4, ChatGPT, Claude2, and Bard. It is worth noting that GPT-4's self-assessment showed more variability compared to the more balanced evaluations provided by human experts. The most important aspect at this moment is our air superiority with Voidrays. However, we need to be prepared for potential ground-based attacks from the enemy's Roaches and Hydralisks. Our future success will depend on how we can maintain our air dominance while countering the enemy's ground forces. ii. Protoss:For Protoss, keep an eye on Nexus's energy to Chrono Boost important structures. (h) Based on the game situation and strategies used by both sides, provide specific suggestions for the following areas: i. Our Strategy:Propose adjustments to our current strategy to counter the enemy's moves and capitalize on our strengths. ii. Units and Buildings:Offer ways to enhance our unit composition and improve our building layout, suited to the current stage of the game. iii. Economy:Recommend better practices for resource gathering and usage, in line with our strategic needs. iv. Technology:Suggest focused research paths to gain technological advantages, considering our current research status and technology tree. (i) Lastly, consider the current situation and the suggestions provided, make K actionable and specific decisions from the action dictionary. This dictionary comprises four categories of actions: unit production, building construction, technology research, and other actions. Remember to align these decisions with the current stage of the game, and avoid proposing actions that are not currently feasible.We should continue to produce Voidrays and maintain air superiority. However, we should also focus on developing a strong ground army to counter the enemy's Roaches and Hydralisks. 
impact of different prompts: 
 In evaluating the Chain of Summarization method using the GPT3.5-turbo-16k model in TextStar-Craft II, we analyzed the effects of two different prompt types on LLM agent performance when playing as Protoss against Zerg. The results, detailed in Table 4 , showed a notable improvement in performance with more complex prompts. Simple Thought Chain: Using a basic prompt (see Prompt 1), the LLM agent could perform elementary operations like worker production, base establishment, and basic combat unit production. However, this approach was limited in developing advanced strategies, such as research upgrades or comprehensive game analysis, indicating a narrower strategic depth with simpler prompts.Complex Thought Chain: A more intricate prompt (see Prompt 2) guided the LLM agent through a series of critical phases, including situation overview, analysis, strategic planning, and decisionmaking. This comprehensive approach enabled the agent to engage in advanced strategies like research upgrades, tech tree exploration, and complex military maneuvers. It proved particularly effective against higher difficulty levels (e.g., "Harder" lv 5 ), showcasing enhanced strategic capabilities.Our analysis underscores the importance of complex prompts that replicate the thought processes of seasoned StarCraft II players. These advanced prompts are essential for LLMs to fully understand and strategically engage in the sophisticated aspects of the game.Our analysis reveals a stark contrast in decision-making between AlphaStar and our LLM agent. While AlphaStar demonstrates superior micromanagement skills, it occasionally lacks rationality in its strategic choices. Conversely, the LLM agent consistently exhibits logical decision-making, as evidenced by its proactive anticipation of threats and strategic planning, detailed in Appendix E and Appendix F. Anticipating Threats: Figure 3 illustrates a stark contrast between AlphaStar and the LLM agent in their ability to anticipate and respond to potential threats. In Figure 3 .a, AlphaStar overlooks the impending danger posed by enemy Oracles, failing to recognize the need for adequate defensive structures. This oversight leaves AlphaStar's base vulnerable to a devastating attack, highlighting its inability to proactively assess and mitigate potential risks. Conversely, in Figure 3 .b, the LLM agent demonstrates remarkable foresight by analyzing the enemy's strategy and recommending appropriate defensive measures. The agent's log data at 05:00 game time reveals its ability to infer the enemy's Roach-centric strategy based on the presence of key structures such as the Roach Warren and Spawning Pool. Armed with this insight, the LLM agent provides a structured decision-making framework to address the identified threat:• Enemy's Strategy: The enemy has established a Hatchery, Roach Warren, Extractor, and Spawning Pool. This indicates a potential strategy centered around Roach production and early aggression. • Given the enemy's potential for early aggression with Roaches, we should prioritize defensive structures such as Photon Cannons and Shield Batteries. Additionally, consider scouting the enemy base to gather more information about their strategy. • Decisions:0: <BUILD PHOTONCANNON> 1: <BUILD SHIELDBATTERY> Flexible Unit Composition: AlphaStar often adheres to fixed unit compositions, leading to ineffective strategies in dynamic scenarios (Figure 13 .a-d). In contrast, the LLM agent demonstrates adaptability in unit selection, effectively countering varied enemy tactics (Figure 13 .e-h). For instance, in Figure 13 .f, the Zerg (Built-in AI) employs a Hydra-Roach composition. The LLM agent, playing as Protoss in 09:00, recognizes the need to adapt its strategy based on the enemy's groundbased army. The agent's thought process, as shown in the log, reveals its ability to analyze the situation and make informed decisions:• Enemy's Strategy: The enemy seems to be focusing on a ground-based army composition, consisting of Roaches, Swarm hosts and Hydralisks. • Key Information: The most important aspect at this moment is our need to expand our unit composition and technology tree to counter the enemy's strategy effectively. We should prioritize unlocking advanced units and upgrades to gain an advantage. Consider researching psionic storm at the Templar Archives to deal with the enemy's ground units effectively. • Decisions: 4: <RESEARCH PSISTORMTECH> The LLM agent's superior performance in threat anticipation and unit composition adaptability stems from its structured, transparent decision-making process. By analyzing the situation, identifying key information, and making informed decisions based on the evolving game state, the LLM agent demonstrates a blend of human knowledge and logical reasoning. This approach enhances the interpretability of the agent's gameplay, facilitating better collaboration and strategic adaptation in complex scenarios, ultimately leading to more successful outcomes compared to AlphaStar's opaque strategies.In 2017, "StarCraft II" became free-to-play, allowing players to access the original "Wings of Liberty" campaign and the multiplayer modes for free, which helped maintain the game's player base. The legacy of "StarCraft II" in the gaming community is marked by its technical and strategic depth, the skill it demands from its players, and its significant impact on the broader e-sports industry.Claude2: Absolutely, here's an overview of StarCraft II:-Races: There are three playable races in StarCraft II -the Terrans (humans), the insectoid Zerg, and the technologically advanced Protoss. Each race has distinct units, structures, abilities, and playstyles.-Terrans are versatile and mobile, with units like Marines, Tanks, and Battlecruisers. They can adapt quickly and have a lot of defensive and zone control options.-Zerg are focused on overwhelm and expansion, capable of producing large armies rapidly. Key units are Zerglings, Roaches, Hydralisks and Mutalisks. They can overrun opponents through sheer numbers.-Protoss have advanced technology and psionic abilities. Their units like Stalkers, Immortals and Carriers are slower but more powerful. They rely on coordination and positioning in engagements.-eSports Scene: StarCraft II has a longstanding professional esports scene in Korea especially. Top players compete in premier tournaments like GSL and Proleague for prize money and glory.-Popular pro players include Serral, Maru, Rogue, Dark and soO. Blizzard directly supports the scene through the ESL Pro Tour and other circuits.-Game Modes: The main competitive mode is 1v1, but 2v2, 3v3 and 4v4 team games are also popular. There is also a Co-op PVE mode where players team up against AI enemies. Custom games allow for all kinds of player-created variants.-Expansions: Wings of Liberty, Heart of the Swarm and Legacy of the Void are the three installments that comprise StarCraft II. Each focuses on one race's campaign story while expanding multiplayer.Let me know if you need any clarification or have additional questions! Bard: Sure. Here is an overview of StarCraft II, including its races, esports scene, and other pertinent details:Races:StarCraft II features three playable races: the Terrans, the Zerg, and the Protoss. Each race has its own unique units and abilities, and the player must use their strengths to defeat their opponents.• Terrans: The Terrans are a versatile race that combines human ingenuity with advanced technology. They have a wide variety of units to choose from, including Marines, Siege Tanks, and Battlecruisers. Terrans are also known for their powerful mobile structures, such as Command Centers and Factories. • Zerg: The Zerg are a swarm-based race that overwhelms their enemies with sheer numbers.They have a variety of cheap and expendable units, such as Zerglings and Roaches, as well as more powerful units such as Hydralisks and Ultralisks. Zerg players are also known for their ability to quickly expand their bases and produce units. • Protoss: The Protoss are a highly advanced race that relies on powerful technology and psionic abilities. They have a limited number of units to choose from, but each unit is very powerful. Protoss players are known for their strong economies and their ability to build powerful armies.StarCraft II has a large and dedicated esports scene. StarCraft II tournaments are held all over the world, and some of the best players in the world can earn millions of dollars in prize money. Zerg:• Creep Spread: Zerg players can spread "creep" across the map. Creep is a slimy substance that not only provides vision but also enhances the movement speed of Zerg units when they are on it. Creep is generated by buildings called "Creep Tumors" and the primary Zerg building, the Hatchery or Lair.• Larva Mechanic: Zerg units are produced from Larvae, which are spawned by Hatcheries and Lairs. The number of Larvae determines how quickly a Zerg player can build units. Managing Larvae effectively is crucial to Zerg gameplay.• Biological Units: The Zerg rely heavily on biological units and are known for their ability to quickly swarm opponents with large numbers of units. Their units often have regenerative abilities and can be upgraded to various forms. Evolution Chambers: Zerg units can be upgraded at Evolution Chambers, allowing players to adapt their units to counter their opponent's strategies.Protoss:• Warp-In Mechanic: Protoss units can warp in directly onto the battlefield from Pylons or Warp Gates, providing a unique mobility advantage. However, this requires strategic placement of Pylons and Warp Gates.• Shields: Many Protoss units have energy-based shields in addition to their regular health. Shields regenerate over time, making hit-and-run tactics effective when used wisely.• Psionic Abilities: Protoss units often have psionic abilities that can be used for offense or defense. Examples include Force Fields, which can block enemy units, and Psionic Storm, which deals area damage.• Warpgate Research: Protoss players can research the "Warpgate" technology, allowing them to transform their Gateways into Warp Gates, significantly increasing their flexibility in producing units.Terran:• MULEs and SCVs: Terran players can call down MULEs (Mechanized Utility Drones) to quickly gather resources. SCVs (Space Construction Vehicles) can also repair buildings and mechanical units.• Building Mobility: Terran buildings are often mobile, with the ability to lift off and land in new locations. This flexibility allows for repositioning and strategic play.• Supply Depots and Wall-offs: Terran players use Supply Depots to provide supply and can use them to create wall-offs to block enemy movement.• Terran Infantry: Terran infantry units can be upgraded with a variety of weapons and equipment, making them versatile and adaptable to different situations.• Siege Tanks: Terran players often employ Siege Tanks, which can transform into siege mode to deal massive area damage. This unit is a cornerstone of Terran defensive strategies.Each race in StarCraft II offers a distinct and challenging gameplay experience, and mastering their unique mechanics is essential for success in the game. Players often choose a race that suits their preferred playstyle, whether it's the swarm tactics of the Zerg, the advanced technology of the Protoss, or the adaptability of the Terran.GPT4: "StarCraft II" stands out in the RTS genre due to its distinct, asymmetric gameplay, meaning each race has unique mechanisms, units, and strategies that create a deep, strategic environment.Here's a detailed look at the specific racial mechanics for the Zerg, Protoss, and Terran factions:1. Zerg:• Creep: One of the most distinctive aspects of the Zerg is their reliance on Creep, a living mat of bio-organic substance that spreads from their main structures. Zerg buildings, except for the Hatchery, must be built on Creep. It also allows Zerg units to move faster when they are on it, providing both strategic and defensive advantages. • Larvae and Macro Mechanics: Zergs uniquely produce units through Larvae generated by their primary structure, the Hatchery (which can be upgraded to a Lair and then a Hive for more advanced units). These Larvae morph into different units, meaning the Zerg can adapt their production in real-time as they see fit. The Queen unit can also spawn additional Larvae to speed up production, emphasizing the Zerg's swarm-oriented strategy. • Rapid Expansion: Zerg players often rapidly colonize areas of the map to gain more resources, using inexpensive units to overwhelm opponents. Their ability to quickly spread Creep helps in establishing new bases and fortifying territories.2. Protoss:• Warp Gate Technology: A definitive feature for the Protoss is their advanced technology, represented through the transformation of their Gateway buildings into Warp Gates. This allows the production and immediate deployment of units across the battlefield, provided there is a power field (generated by structures like the Pylon). This ability is crucial for both reinforcing units during a battle and executing surprise attacks. • Shields and Health: Protoss units and buildings have two layers of health: a regenerative shield and their actual health. This characteristic allows Protoss armies to be very resilient, as shield points regenerate over time when not under attack. • Chrono Boost: The Protoss Nexus, the main base structure, has an ability called Chrono Boost that can be used to speed up the production of units, research, and buildings. This feature is integral to the race's adaptability, allowing them to respond quickly to threats. 
discussion: 
 TextStarCraft II enhances the use of LLMs for strategic decision-making in StarCraft II, showcasing their capabilities in adaptive strategy and crisis management. However, the frameworks reliance on rule-based scripts for micro policies, limitation to non-visual data, and a subset of the game's races may restrict the diversity and applicability of AI strategies. Additionally, resource limitations have constrained the performance capabilities of our system. Despite these challenges, TextStarCraft II establishes a new benchmark in RTS games, promoting deeper AI-human collaborative research. Future improvements will focus on integrating visual inputs, expanding race support, and optimizing resource usage to enhance strategic complexity and performance against established AI models. 
acknowledgements: 
 This work was supported by the StarCraft II AI 5 and player communities. We extend our gratitude to Burny 6 for providing detailed documentation and offering warm-hearted replies during the development process. We are grateful to several StarCraft II professional and amateur players who participated in our testing. Special thanks go to:• Zhiyao(aka. HistTATP)Wang 7 Their expertise and insights were invaluable to our research.This work introduces a novel method for addressing real-time strategy (RTS) tasks and strategic missions using Large Language Models (LLMs), expanding their application beyond traditional domains. Our approach enhances the interpretability of decisions made by LLMs, offering insights into the language policies that govern AI behavior in complex scenarios. These advancements have potential implications for sectors requiring nuanced decision-making and strategic planning, further bridging the gap between AI capabilities and human-like reasoning in dynamic environments.This paper addresses both the positive and negative societal impacts of employing Large Language Models (LLMs) for real-time strategy gaming environments, such as StarCraft II. On the positive side, enhancing interpretability through language-driven strategies fosters greater understanding and transparency in AI decision-making, which can be beneficial across various sectors that require complex strategic thinking. However, the potential negative impacts include the risk that AI-generated strategies or recommendations might be misleading. This could adversely affect human decisionmaking, particularly in scenarios where users rely heavily on AI suggestions without sufficient oversight or understanding. Such misleading advice could lead to suboptimal or even harmful decisions in critical areas like security or management. To mitigate these issues, we advocate for rigorous validation of AI advice, comprehensive user education on the AI system's capabilities and limitations, and the development of fail-safes that can alert users to potential inaccuracies or biases in AI-generated strategies.A.1 Experiment Setup Agent and Opponent Selection: To ensure a consistent and controlled experimental environment, LLM agents are configured to play as Protoss against Zerg AI opponents. This setup allows for a systematic evaluation of strategic capabilities across a range of difficulties. The difficulty levels are defined as follows to cover a comprehensive spectrum of challenges: Parameter Settings: The temperature parameter is fixed at 0.1 to focus on strategy-driven actions over randomness.EQUATIONGame Version: Our experiments were conducted across three different versions of the game to ensure robustness and applicability of the results. The versions tested include Patch 5.0.11, Patch 5.0.12, and Patch 5.0.13.Building on StarCraft II's established player performance analytics, our evaluation framework in TextStarCraft II integrates these insights with custom adaptations to comprehensively assess LLM agents gameplay strategies.Win Rate: The most crucial metric, Win Rate directly reflects the agent's capability and performance within the environment. It is calculated as the percentage of games won by the agent out of the total games played.Population Block Ratio (PBR): PBR is a measure of the agents macro-management effectiveness, particularly in resource allocation and population growth. It is defined as:EQUATIONHere, the metric calculates the ratio of time spent at population capacity to the total game time until the agent first reaches maximum supply (200/200). A higher PBR indicates poorer macro-strategic decision-making and ineffective planning.Resource Utilization Ratio (RUR): RUR evaluates the agent's efficiency in resource management throughout the game. It is calculated as: RUR = Total Minerals + Total Gas Used Game Duration (6) This metric assesses the total resources used against the duration of the game until the agent first reaches maximum supply. A higher RUR signifies underutilization of resources, reflecting weaker macro-strategic capabilities.Average Population Utilization (APU): APU measures the efficiency in utilizing the available population capacity. It is calculated as:EQUATIONThe metric averages the ratio of used population to population cap until the agent first reaches maximum supply. A higher APU suggests better utilization of population and more effective macromanagement.Technology Rate (TR): TR assesses the extent to which the agent explores and utilizes the technology tree. It is defined as: TR = Completed Technologies Total Technologies Available (8) This metric calculates the proportion of completed technologies and buildings to the total available, from the start to the end of the game. TR indicates the agents tendency towards technological advancement and is not necessarily indicative of better or worse performance.Data Collection and Analysis: Due to technical constraints with the sc2reader library for AIgenerated replays, we utilized gameplay log data for in-depth analysis. This approach ensures a thorough and precise evaluation based on our tailored metrics.Training Phase: To fine-tune open-source Large Language Models, we utilized two NVIDIA A100 40GB GPUs, providing the necessary computational power for extensive model training. The entire experiment took approximately 70 hours.Testing Phase: The development and testing of these fine-tuned models were performed using two NVIDIA A100 40GB GPUs. For running the StarCraft II environment, we employed an NVIDIA GeForce RTX 3060 GPU paired with a 13th Gen Intel(R) Core(TM) i5-13400F processor operating at 2.50 GHz, ensuring smooth execution of the game simulations.In the course of our research, we engaged 30 StarCraft II masters and grandmasters, including professional and semi-professional players, to participate in our experiments. To safeguard their privacy, we have chosen not to disclose their personal information in this paper.Each participating player was compensated with 15 USD for their involvement in the study. The latency in our system is primarily determined by the inference speed and model size of the LLM. By leveraging fine-tuned open-source LLMs, our approach can achieve real-time feedback. Our Chain of Summarization (CoS) method is crucial in enabling this real-time feedback, significantly outperforming traditional CoT methods in terms of responsiveness.For instance, with CoS, even smaller LLMs like the fine-tuned Qwen-2 7B can achieve real-time interaction (100 ms/step) and demonstrate competitive performance against human players. This level of real-time capability is not possible with standard CoT methods, highlighting the effectiveness of our CoS approach in facilitating swift strategic decision-making in complex, dynamic environments.Although larger models such as GPT-4, Claude, and Gemini still face challenges in achieving realtime feedback (1s/step) due to their size, complexity, and network transmission factors, our CoS method significantly reduces their response times compared to CoT. This improvement allows for near-real-time performance in many scenarios. We are continuously optimizing our system to further enhance the real-time capabilities of these larger models.Below are the delays observed for each step when different LLMs interact with the CoS method:Model Type Model Name Delay (each step) There are two types of system prompts in the 7.1 section. Lastly, consider the current situation and the suggestions provided, make {K} actionable and specific decisions from the action dictionary protoss_action_dict . This dictionary comprises four categories of actions: unit production, building construction, technology research, and other actio ns. Remember to align these decisions with the current stage of the game, and avoid proposing actions that are not currently feasible. Finetuned Open-Source LLM Qwen2-7b 98ms Finetuned Open-Source LLM Qwen2-1.8b 64ms Finetuned Open-Source LLM LLAMA2-7B 102ms Closed-LLM GPT3.5 1.2s Closed-LLM GPT4 2.3s Closed-LLM Gemini-PRO 0.3s 
 
 Appendix:  


Decision: The reviewers and I agree that this paper makes a solid contribution to benchmarking for LLMs. While the paper is light on theoretical contributions, it is an excellent piece of empirical work and will likely be of interest to a wide variety of researchers.

As a side note, I sincerely hope you're working on trying this with visual data using a multi-modal LLM!

Reviews: review: paper_summary: paper_summary: This paper introduces TextStarCraft II, a benchmark designed to assess long-term strategic decision-making in the context of playing StarCraft II. It models the game-playing process through pure textual representation and also develops a chain-of-summarization pipeline to aid LLM's decision-making to achieve victory. They evaluate various LLMs under different prompting strategies against the built-in AI mode of StarCraft II. The experimental results indicate that prompting powerful closed-source LLMs (e.g., GPT-4) and fine-tuned open-source LLMs (e.g., QWEN-7B) can perform competitively with the Level 5 (the highest level) built-in AI of StarCraft II.
strength_weakness: strength_weakness: - The objective of how LLMs achieve victory in the game is not clearly defined.

- The process by which LLMs execute actions and make observations within the textual setting is insufficiently explained.
questions: questions: The paper introduces a benchmark for evaluating large language models' (LLMs) long-term strategic decision-making capabilities in the context of playing StarCraft II. This benchmark models gameplay in a purely textual environment and includes the design of a chain-of-summarization pipeline to aid LLMs in playing and winning the game. The topic is interesting and beneficial to the LLM research community. However, I have several concerns, primarily about the overall pipeline setting and the mechanics of how LLMs engage with the game.
- The objective of optimizing LLMs through prompting or fine-tuning to win the game appears to be inadequately defined. It remains unclear what the specific winning criteria for LLMs are and how these models compete against the built-in AI to meet these criteria.
- The methodology detailing how LLMs execute actions and observe the environment in each round lacks clarity. According to Figure 1, each round involves the LLM making decisions regarding necessary actions and translating multi-frame observations into text. However, it is not explicitly described what types of actions are executed or how multi-frame observations are represented within the purely textual environment.
- The experimental results indicate that well-optimized LLMs, with appropriate prompting, can perform competitively with built-in AIs. It would be beneficial to provide deeper insights.
limitations: limitations: N/A
paper_summary: paper_summary: The authors introduce TextStarCraft II, a framework for transferring the state of the Real Time Strategy (RTS) Starcraft II (SC2) into text form and Chain of Summarization (CoS), an extension of the traditional Chain of Thought (CoT) style prompting for condensing information and accelerating LLM inference. The authors demonstrate their system is capable of defeating gold-level, human players in SC2 in real time matches.
strength_weakness: strength_weakness: A weakness of the authors’ approach is that there is a high likelihood that most modern LLMs were trained on text that included comprehensive strategy discussions of SC2 in text form. However, this is mostly mitigated by the authors’ investigation of both open and closed-source LLMs in section 5.1. However, it would still be interesting to see if these results still hold in newer RTS games that are likely to not be found in most LLMs’ training distribution. 

A lesser weakness the reviewer would like to suggest to the authors is that it may be worth considering staying away from Starcraft-specific terms in order to reach a wider audience that may be less familiar with the Starcraft series as a whole.
questions: questions: 1.) When playing against human players, where any restrictions placed on availability of units or maps?

2.) Do the authors believe the results would generalize across the RTS genre and has there been any investigation into this?

3.) To what extent were the macro and micro actions predefined?
limitations: limitations: The authors briefly touch on some limitations of their work, however it is unclear whether the topics addressed in the questions were not included due to being outside the scope of the work, or not relevant given some metric or detail the reviewer may have missed.
paper_summary: paper_summary: This paper introduces TextStarCraft II, a text-based environment to evaluate the strategic decision-making and planning capabilities of large language models (LLMs) in real-time scenarios within StarCraft II (SC2). The study addresses the limitations of traditional Chain of Thought (CoT) methods by proposing the Chain of Summarization (CoS) method, which enhances LLMs’ abilities to process complex information and make strategic decisions efficiently. Key experiments include testing various LLMs against SC2's built-in AI, evaluating commercial models on SC2 knowledge, and conducting human-AI matches to assess performance and strategic adaptability.
strength_weakness: strength_weakness: - Human Interaction Assurance: The paper does not clearly explain how LLMs interact with the game. From lines 120-121, it seems that human players might use LLM suggestions to operate the game. If this is the case, how do you ensure that players fully adhere to the LLM's suggestions?
- Latency and Real-Time Feedback: In Section 5.3, detailed information on latency is not provided. Can your method provide real-time feedback?
- Resource Dependency: The reliance on rule-based scripts for micro-management and the limitation to text-based inputs may restrict the diversity and applicability of AI strategies.
questions: questions: N/A
limitations: limitations: N/A

Comments: comment: Thank you for your explanations. I'll keep my current score.
Thanks for the rebuttals. My concerns are mainly addressed. I will raise my score to 6.
Thank you for the authors' response.   
We have reviewed your feedback and noted that most of our concerns have been addressed.