Large Language Models Play StarCraft II:Benchmarks and A Chain of Summarization Approach
Authors: Weiyu Ma
Qirui Mi
Yongcheng Zeng
Xue Yan
Runji Lin
Yuqiao Wu
Jun Wang
Haifeng Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: LLM Agent
StarCraft2
Primary area: reinforcement_learning
TL;DR: large language models play starcraft2

Abstract: With the continued advancement of Large Language Models (LLMs) Agents in reasoning, planning, and decision-making, benchmarks have become crucial in evaluating these skills. However, there is a notable gap in benchmarks for real-time strategic decision-making. StarCraft II (SC2), with its complex and dynamic nature, serves as an ideal setting for such evaluations. To this end, we have developed TextStarCraft II, a specialized environment for assessing LLMs in real-time strategic scenarios within SC2. Addressing the limitations of traditional Chain of Thought (CoT) methods, we introduce the Chain of Summarization (CoS) method, enhancing LLMs' capabilities in rapid and effective decision-making. Our key experiments included:
1. LLM Evaluation: Tested 10 LLMs in TextStarCraft II, most of them defeating LV5 build-in AI, showcasing effective strategy skills.
2. Commercial Model Knowledge: Evaluated four commercial  models on SC2 knowledge; GPT-4 ranked highest by Grandmaster-level experts.
3. Human-AI Matches: Experimental results showed that fine-tuned LLMs performed on par with Gold-level players in real-time matches, demonstrating comparable strategic abilities.

All code and data from this
study have been made pulicly available at https://github.com/histmeisah/Large-Language-Models-play-StarCraftII

Large Language Models Play Star II: Benchmarks and A Chain of Summarization Approach
Weiyu Ma Qirui Mi Yongcheng Zeng'. 1.2 Xue Ya Yuqiao Wu!.?, Runji Lin Haifeng Zhang 1.2.4 Jun Wang"
Institute of Automation, Chinese Academy of Sciences, China 2 School of Artificial Intelligence, University of Chinese Academy of Sciences, China Al Centre, Department Computer Science, UCL Nanjing Artificial Intelligence Research of IA, China
Abstract
With the continued advancement of Large Language Models (LLMs) Agents in reasoning, planning, and decision-making, benchmarks have become cru- cial in evaluating these skills. However, there is a notable gap in benchmarks for real-time strategic decision-making. StarCraft II (SC2), with its complex and dynamic nature, serves as an ideal setting for such evaluations. To this end, have developed TextStarCraft II, specialized environment for assess- ing LLMs in real-time strategic scenarios within SC2. Addressing the limita- tions of traditional Chain of Thought (CoT) methods, introduce the Chain of Summarization (CoS) method, enhancing LLMs' capabilities in rapid and ef- Our key experiments included: 1. LLM Evaluation: Tested 10 LLMs in TextStarCraft II, most of them defeating LV5 build-in AI, showcasing effective strategy skills. 2. Commercial Model Knowledge: Eval- uated four commercial models on SC2 knowledge; GPT-4 ranked highest by Grandmaster-level experts. 3. Human-Al Matches: Experimental results showed that fine-tuned LLMs performed on par with Gold-level players in real-time matches, demonstrating comparable strategic abilities. All code and data from this study have been made pulicly available at https:// com/ http/hssteissal -odeeeoooooeosetttaaaaaee play-
fective decision-making
Models
1 Introduction
Real-time strategy decision-making and long-term planning are critical Al challenges, necessitating rapid, tactical decisions and strategic adaptability over time. StarCraft (SC2), one of the world's most popular and challenging e-sports, exemplifies these demands through its dynamic gameplay. Players must manage resources, construct bases, and command armies while making quick deci- sions and adapting their long-term strategies to evolving battlefield conditions. The game's layered gameplay spans economic management, military strategy, and tactical execution, making SC2 a valuable model for AI research, particularly in reinforcement learning (RL). SC2's complexity, real- time nature, and the fact that it is considered one of the hardest games in the world pose significant challenges for AI systems, requiring them to master various aspects of the game simultaneously. Further details about StarCraft can be found in the appendix ZI Pioneering efforts, exemplified
Coorespoondecce to saiffnn.zhhange @iw.ac.cn ), (jun wang www.wwa..nn.a.......
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

by DeepMind's AlphaStar [24]], have demonstrated significant advancements in this domain, show- casing AI's growing proficiency in strategic gameplay. With the evolution of LLMs areas like reasoning, planning, and decision-making, these models have begun to show potential in tasks tra- ditionally dominated by RI. approaches(33], Benchmarks such as AGENTBENCH [[5] have been instrumental in evaluating these capabilities in multi-turn, open-ended contexts. However, despite these developments, a specific benchmark for assessing LLMs' capabilities in real-time strategy decision-making and long-term planning in environments like StarCraft II is conspicuously absent.
Therefore, we have chosen SC2 as the benchmark for evaluating the real-time strategy decision- making and long-term planning capabiliities of LLMs. Given the lack of language support in existing SC2 environments, we developed TextStarCraft II. Utilizing the python-sc2 framework, TextStar- Craft II converts the complex gameplay dynamics of SC2 into an interactive, text- based format. The python-sc2 interface @ is utilized convert game data into text, enabling L.LM agents to perform macro-strategic actions through language commands. For micro-strategic actions, we implement a rule-based approach akin to that used by OpenAl Five [Z]], employing predefined Python scripts. This allows LLM agents to engage in competition against the game's built-in AL, other LLM agents, and human players through the execution of these scripted actions. Addressing the challenges posed by the intricate decision-make method. This approach enhances the capacity of LLM Agents in processing complex information and making strategic decisions by incorporating single- frame and multi-frame summarization mod- ules, each aiding in understanding the immediate game state and processing sequential data for strategy formulation and decision-maker making..
process in SC2, we propose the Chain of Summarization
In this study, we conduct a thorough exploration of LLMs' application and effectiveness within SC2 via the TextStarCraft II environment. Our experimental framework includes assessing the CoS method, evaluating the performance of proprietary and fine-tuned open-source LLMs in TextStar- Craft II, testing real-time human-computer interaction, and using StarCraft II-themed question- answering tasks evaluated by human experts. We also analyze the impact of varying prompts on LLMs, their strategic preferences, and the interpretability of their decision-making processes.
Our contributions are manifold:
TextStarCraft II Development: We present TextStarCraft II, novel environment that not only enables the evaluation of LLMs in strategic gaming contexts but also supports real-time human-computer interactions.
Chain of Summarization Method and Agent: By introducing the CoS method and re- leasing an open-source agent, we offer powerful example and a high-level interface for the community, fostering further development and interaction with TextStarCraft II.
Diverse Evaluations of LLMs: Our extensive evaluations encompass testing LLMs against the game's 's built-in AI, assessing their understanding of StarCraft II through ex- perts reviews, and conducting human-Al matches. These methodologies underscore LLMs' proficiency in strategic decision-make making and their potential for human-like gameplay.
2 Background: StarCraft II
StarCraft II (SC2) is a real-time strategy (RTS) game developed by Blizzard Entertainment, renowned for its strategic depth and complexity. features three distinct playable races: Terrans (humans with advanced technology), Protoss (a psionic species with powerful abilities), and Zerg (a biologically advanced, insectoid race), each with unique units, structures, and gameplay mechanics. The game's core elements include resource management, base construction, unit production, and tactical combat. Players must efficiently gather resources (minerals and vespene gas), construct and expand bases, compose balanced armies, and execute strategic plans while adapting to opponents' moves. This multi-faceted gameplay requires players to make rapid decisions under incomplete information, balancing short-term tactics with long-term strategy. SC2's complexity is further en- hanced by asymmetric design, where each race has distinct strengths and weaknesses, leading to a rich variety of strategies and counter-strategies The game also incorporates elements of economics, technology progression, and territorial control, making it a comprehensive test of strategic thinking and multitasking ability.

---

Since its release in 2010, SC2 has been cornerstone of competitive gaming, featuring prominently in major e-sports tournaments worldwide. The game's professional scene is characterized by high- level play, particularly from South Korean competitors, though it maintains a strong international presence. SC2's 8 strategic complexity has made it subject of extensive study, not only within the gaming community but also in academic fields such artificial intelligence and cognitive science. Researchers have utilized SC2 as a testbed for developing and evaluating Al algorithms,, particularly in areas of hierarchical planning, multi-agent coordination, and decision-making under uncertainty. The game's s rich strategic landscape, combined with its clear victory conditions and quantifiable performance metrics, makes it an ideal environment for benchmarking Al systems against human expertise in complex, real-time domains. Moreover, SC2 has been used to study human cognition, particularly in areas of attention allocation, decision- making under time pressure, and skill acquisi- tion. The game's 's replay system, which records every action taken by players, provides researchers with wealth of data for analyzing human performance and strategy evolution over time.
3 Related Work
StarCraft II Full Game AI: StarCraft Al research, initially focused on StarCraft with develop- ments like BiCNet [19] for multi-agent coordination, has significantly advanced in the StarCraft era. The release of PySC2 [23]] by DeepMind, coupled with Blizzard's game replays, propelled this research field. A key breakthrough was AlphaStar [[a]], which achieved Grandmaster level and defeated top players, demonstrating the potential of RL in complex environments.
Subsequent research expanded upon these foundations. Mini-AlphaStar [B] simplified input vari-
ables without compromising learning effectiveness. TG [0] and HierNet-SC2 [044]] explored effi-
cient RL strategies, with the latter bypassing supervised pre-training. AlphaStar Unplugged [[G]]
represented a leap offline RL using human replays. TStarBotsX [ and SCC [26] furthered feder-
ated learning approaches, achieving notable success against master and grandmaster level players.
Recent advancements include DI-star D, which is accessible for home computer deployment, and ROA-Star [9]. enhancing AlphaStar's ' training framework with goal-conditioned exploiters and re- fined opponent modeling techniques. ROA-Star's practical tests against professional players have shown impressive results, marking significant progress in real-time strategy Al.
LLM Agent and Benchmark: The introduction of GPT3.5 [[ has significantly propelled the re- search on LLM agents forward. Projects such as React [29]] and AutoGPT G laid the groundwork for more sophisticated implementations. Within the MineDojo environment [5] initiatives like GITM [34]] Voyager [25], and others [6. 28, [0] have underscored LLM agents' adaptability to a variety of tasks and expansive open-world scenarios. Additionally, environments like TextWorld [[]] and ALFWorld [20) enrich agent training by integrating text-based strategy and action execution across simulated and visual contexts, facilitating advanced generalization and adaptive learning. Addi- tionally, optimization methods such as Reinforcement Learning with Human Feedback (RLHF) have also improved the performance of large language models.
Further advancements in multi-agent coordination and virtual social dynamics have been achieved through MetaGPT [8], Camel [[]]] and Generative Agents [[8]. Benchmarking platforms such as AGENTBENCH play a critical role in evaluating these developments, with AGENTBENCH exam- ining dlecision-making in comprehensive, open-ended contexts.
In StarCraft II, despite the development of advanced Al agents, there remains a gap in evaluating LLMMs, especially in real-time strategy and long-term planning. This led to the creation of TextStar- Craft II, an environment tailored for testing LLMs in these specific aspects, filling a critical need for natural language interaction capabilities in AI research.
4 TextStarCraft II
TextStarCraft II provides text-based interface for LLMs within the SC2 environment, utilizing the python-sc2 framework to translate complex gameplay into text. Key components include the Observation-to-Text Adapter and the Text-to-Action TextSarrrettt stands out from
3

---

other text-based environments like WextWorld and ALFWorld due to its more complex and dynamic gameplay. It requires agents to manage multiple aspects such as resource allocation, base building. and military strategy in real-time, challenging both their adaptability and decision-making. Addi- tionally, the environment demands advanced language understanding to interpret and execute more open-ended and diverse commands, enhancing the need for sophisticated natural language process- ing capabilities. We will introduce the main components of TextStarCraft below.
TextStarCraft II's observation space is designed to equip LLM agents with essential game insights, effectively navigating the fog of war StarCraft II. The observations encompass six key categories:
Observation
Resources: Game resources and supply levels. ("Minerals: 500, Vespene Gas:200 ")
Units: Types and quantities the players units.(c.g., "15 Zealots, 5 Carriers")
Buildings: Information on the players (uiddings.(e.g.. Gateway, Nexus")
In-Process Activities: Ongoing construction and production (e.a. "Researching Warp- gate: 70
Enemy Status: Visible enemy units and buildings
Research Progress: Updates on the player's technological advancements.
This structured approach in the observation space enables LLM agents efficiently process and utilize vital game data for strategic decision-makee in TextStarCraft II.
Action There are mainly two types of actions: Macro Actions and Micro Actions.
Macro Actions:
Covering broad strategic decisions such Training Units, Building Struc- tures, Researching Technologies, and Other Strategic Maneuvers. Examples include:
Building construction (e.g.. "Build Cybernetics Core")
Unit production (c.g.. "Train 5 Probes (Protoss worker)")
Research (e.g., "Research Blink Tech")
Tactical maneuvers (e.g.. "Scout with 1 probe", "Attack enemy base")
Micro Actions: Script-managed for precise placements and targeting, not directly con- trolled by the agent.
Reward The reward function R is crucial for aligning agent behavior with the game's objectives, assigning values of (-1,0, 1} based on losing, drawing, or winning a match.
Game Modes TextStarCraft II enriches strategic gameplay with several modes. The Built-in AI mode features 10 difficulty levels, ranging from VeryEasy(LVI) to CheatInsanc(LV10) and incorpo- rates six strategic styles: RandomBuild, Rush, Timing. Power, Macro, and Air. The Agent AI mode enables competition against rule-based Als and other LLM agents. The Human mode supports interactions with real-world players, enhancing gameplay realism.
Victory Conditions our TextStarCraft2 environment, victory conditions align with those of standard StarCraft II 1v1 game, akin benchmarks such AlphaStar, ROA-Star, DI-Star. Victory requires the LLM agent to destroy all opponent structures, standardizing our benchmark with practices in various StarCraft II AI competitions.
Chain of Summarization
The Chain of Summarization (CoS) method, integral to the TextStarCraft II framework, draws inspi- ration from computer hardware's 's cache mechanisms and RL's frame skipping techniques. Serving as an euhancement traditional Chains of Thought (CoT) [27]] and as standard plug-in, CoS refines strategic decision-making makiin in StarCraft II through:
Information Compression: It focuses on key data, reducing overload and sharpening strategic clarity.

---

TextStarCraft2
Obs to Text
STARTRAFT
Observation
Text to Action
queue
Syslem prompt
Situation Overviaw
User example
Single frame
Action
Stussion Analyyiss
queue
Strategic Planning
LLM rule-based
Opponent Strategy
Analysis
Strallgic
Decision Making
L1 Output
Analysis
Suggestions
Decisions
LLM
Chain of Summarization
Figure 1: Interacting with LLM using the Enhanced Chain of Summarization (CoS) Method in TextStarCraft II. This streamlined LLM--riven gameplay. begins with initialization, where initial game data is converted to text for processing. Next, Single-Frame and Multi-Frame Summa- rization refine and summarize observations into actionable insights using advanced LLM reasoning. In Directive Formulation and Action Scheduling, these insights are segmented into specific actions and queued for execution. The process concludes with Action Retrieval and Execution, where ac- tions are implemented in the game. This cycle continually converts new data into text, enhancing LLM performance in the TextStarCraft II.
Inference Acceleration:
This approach speeds up decision-makinn making by providing a more comprehensive view of the game's 's state.
Global Understanding: CoS equips LLMs with a deeper grasp of game strategies, leading expert-level decisions.
As versatile tool within the TextStarCraft II framework, CoS can operate both as standalone plug- in, enhancing the environment's utility, and as direct interface for user interaction with TextStar- Craft II. This dual functionality not only showcases CoS as an exemplary model for engaging with our environment but also invites further development and customization by the community, broad- ening the scope of strategic Al research in gaming. CoS includes Single-Frame Summarization, Multi-Frame Summarization, and Action Extraction for the Action Queue.
Single-Frans Summarization To make TextStarCraft's raw observation data more comprehensi- ble for LLMs, Single-frame Summarization compresses and extracts key information. This process, denoted as SSF (+), transforms dense TextStarCrafi II observations into condensed form ô, de- scribed by:
Ô Sgr(o).
(1)
There are two approaches to this compression: a language model-based approach using few-shot learning for better alignment with game rules and faster, rule-based approach for extraction and filtering. In our experiments, the rule-based approach is primarily used for quicker interactions.
Multi-Frame Summarization Traditional methods query LLMs at each time step for decision- making ([3), [32]). However, this is inefficient for long-duration games like StarCraft II due to high computation costs and slower LLM inference. Our Multi-Frame Summarization method, inspired by caching in computer hardware and frame skipping in RL, addresses these issues. It synchronizes the quick pace of the game with L.LM processing, ensuring real-time decision-making efficiency and improved comprehension in complex scenarios. Instead of constant LLM querying, we aggregate
5

---

Algorithm Chain of Summarization Interaction in TextStarCraft II
Input: TextStarCraft II game environment ent, Chain length K
1: Set up the environment and obtain the initial raw observation dp env.rese()
2: Initialize the raw observation queue Qohs, action queue Quction and total reward R =
3: Add K instances of raw observation og the raw observation queue Qoba
4: while env is not terminated do
5: if' len(Qobs) 2 K then
6: Initialize Single-Frane Summarization Queue QSFS for raw observation in Qobs do Frame Summarization à (s)
D CoS start
7:
8:
Perform
9:
Add ô the Single-Frame Summarization Queue QSFS
10:
end for Perform Multi-Frame summarization a = -SMr(QsFF)
11:
12:
Apply Chain of Thought reasoning v CoT((r)
13:
extract K actions (a1, 0.. ... .K) = Ex(v)
> CoS end
14:
Add the K actions (a1, a2. (k) to the action queue Qaction if
15:
16:
Obtain the next action a from the action queue Quction Get the reward ra and the next observation 0++1 from the environment Tt, 0++1 = env.step(as)
17:
18:
Add the raw observation 0 to the raw observation queue Qohs
19:
R R + rt
20:
end while
21:
return total reward R
condensed observation information for K steps into a period summary a, described by:
SMr (δ1, .2, ,ūx)-
This method enables comprehensive analysis and strategic planning through a series of steps, includ- ing situation overview, analysis, strategic planning, opponent strategy analysis, suggestion formula- tion, and decision-making. This process is formalized as v, which is the output of CoT reasoning for summarization a, given by:
S == CoT(())
Action Extraction for Action Queue The action queue forms critical link between the Multi- Frame Summarization results, v. and TextStarCraft environment, facilitating communication between the LLM and the game. Within v, key components include analysis, suggestions, and deci- sions. To convert these into actionable steps, we employ regular expression matching and similarity searching in action extractor, donated as Ex(-). This process populates the action queue with actions ready for exccution TextStarCraft II. From the output v CoT reasoning, we utilize the action extractor extract K actions, as formalized in:
(01, 02,,0..... .aa) = Ex(v).
The CoS method optimizes decision-making in TextStarCraft II through streamlined four-stage process: Initially, it sets the initial parameters and transforms the first game frame into text for sub- sequent analysis. Next, it distills key game observations to provide concise snapshot the current situation. Following this, the method translates these summaries into strategic action plans. Finally, implements the planned actions within the game, thereby completing the decision cycle. This process is depicted in Figure II This approach, which updates actions every few frames, effectively manages the fast- fast-paced dynamics of StarCraft II, thereby proving essential for real-time strategic gameplay. The pseudocode is as shown in Algorithm MI
Experiment
our experiment, we detail the setup and key metries (evaluation metries detailed in Appendix A.2. to evaluate macro-strategic decision-making in StarCraft II. We assess the Chain of Summarization's impact on L.M gameplay, compare various L.MMs performance, and evaluate their grasp of Star- Craft II strategies. Our experiments also concludes with human-Al interaction tests.

---

Table I: Performance of L LV5) in TextStarCraft II: Comparing models using either the full CoS or CoS without CoT. Evaluation metrics are elaborated Appendix A.2
MODEL
METHOD
WIN RATE
PBR
RUR
APU
TR
USING FULL CoS
FULL CoS
11/20
0.0781
7875
0.7608
0,4476
GPT4-TUBOR
FULL CoS
12/20
0.0337
8306
0.7194
0.3452
GEMINI-PRO
FULL CoS
5/20
0.0318
9284
0.6611
0.3571
GL.M4
FUI.I. CoS
4/20
0.0327
3131
0.6644
0.2904
CLAUDE2.1
FULL CoS
3/20
0.0219
10867
0.6599
0,4312
USING CoS
WITHOUT CoT
FINETUNE- FINETUNE-CHATCON CHATGLM3 6B
CoS w/o CoT
3/20
0.0528
30356
0.6547
0.1714
FINETUNE- OWEN 1.8B
CoS w/o CoT
8/20
0.0384
12826
0.7506
0.2095
SINTUNE-QWER 7B
CoS w/o CoT
9/20
0.0421
12276
0.7234
0.3214
FINETUNE-LLAMA2 7B
CoS w/o CoT
1/20
0.0469
12295
0.5752
0.0853
6.1 Performance Evaluation of Various LLMs
In this section, we assess the performance of closed-source LLMs [21]], Llama2 70B [22]], and fine-tuned open-source LLMs such as ChatGL.M3 6B [30] and Qwen 1.8B [M]] in the TextStarCraft environment against the built-in AI at level 5, The experimental results are shown in Table M with the evaluation metries detailed in Appendix A We tested closed-source LLMs and the un- fine-tuned Llama2 70B using the standard CoS method. The closed-source models performed well, while Llama2 70B could understand the task requirements and was unable to generate commands based on the given prompts.
Additionally, we fine-tuned open-source models using the entire dataset of GPT3.5-turbo- inter- action logs with TextStarCraft II. Due to computational resource limitations, we removed the CoT component, keeping only the inputs and outputs. The results showed that all evaluated closed-source LLMs were capable of defeating the level built-in AI. The fine-tuned open-source models, despite loss in strategic diversity, still managed to overcome the level AI, predominantly adopting strategy focused on mass-pros mass-produuing stalkers.
Table 2: Performance of models fine-tuned on various datasets in TextStarCraft II, showing win rates(Vss LV5). Datasets are differentiated based on whether they contain all games or only wins, and, for subsets of wins, based on the APU performance percentile.
DATASET
WIN RATE(VS LV
FUL. DATASET (ALI GAMES)
28/100
WINS DATASET (ALL WINS)
48/100
WINS DATASET (BOTTOM 25% 75-100%)
29/100
WINS DATASET (50-75% APU)
37/100
WINS DATASET (25-50% APU)
39/100
WINS DATASET (ToP 25% APU, 0-25%)
54/100
Finally, we investigated the impact of data quality using the Average Population Utilization (APU) metric (detailed in Appendix A.Z) to partition the dataset. The results (Table 2 demonstrated that fine-tuning wins from the top 25% APU games yielded the highest win rate (54/100), while using the full dataset resulted in a lower win rate (28/100). This suggests that training data quality, espe- cially the inclusion of high-performing games, significantly impacts fine-tuned model performance CextStarCraft
6.2 Assessing LLMS' Mastery of StarCraft II Concepts
We evaluated the understanding StarCraft by LLMS like GPT3.5 and GPT-4, focusing on their knowledge of build orders and game mechanics sourced from prominent StarCraft II forums. De-

---

CP%
(a) GPT4 Evaluation of Questions
(b) GPT4 Evaluation of LLMs
Hurnn Dipera LLM Parerrarcs Main Qusetore
G714
(c) Human Evaluation of Questions
(d) Human Evaluation of LLMs
Figure 2: The result of Double-Blind Assessment.
spite a reasonable grasp of basic game dynamics, these models faced challenges with complex elements like the tech tree and supply constraints.
Evaluation Methodology: To gauge the depth of LLMs' StarCraft II knowledge, we tested mod- els, including ChatGPT (GPT3.5), GPT-4, Claude2, and Bard, across five areas: Basic Knowledge, Racial Mechanics, Typical Strategies, Standard Build Orders, and Classic Strategies and Counter- plays (detailed in Appendix G). We used double-blind evaluation, with both Grandmaster-leva human experts and GPT-4 assessing the responses to ensure unbiased scoring.
Both human experts and GPT-4 assessed the LLMs, illustrated in Figures D and Table G. leading to the following insights: In the evaluation of LLMSs on set of complex questions, GPT-4 and ChatGPT demonstrated the best performance, with GPT-4 receiving high scores from both itself and human experts. Claude2 had mixed results, with GPT-4 rating it higher than human experts did. Bard struggled with the complex questions, receiving the lowest scores, especially on questions 3, 4, and 5. Overall, the LLMs were ranked as follows: GPT-4, ChatGPT, Claude2, and Bard. is worth noting that GPT-4's assesaesessmntt showed more variability compared to the more balanced evaluations provided by human experts.
Evaluation Results:
6.3 Human-Al Interaction
Table Match Results: Finetuned Qwen1.8D vs Human.
HUMAN PLAYER
RANK
MMR
RESULT
PLAYER A PLAYER PLAYER C PLAYER
PRO GAMER
5918
0/10
GRANDMASTER
5001
0/10
GOLD
2556
5/10
NEW PLAYER
10/10
Following insights from Section b.Il we evaluated the fine-tuned Qwen1.8B model's real-time inter- action with human players on home PCs, requiring only 4GB of GPU memory. This model faced human players of varied skills from the Asian server, including Grandmaster, a Gold-level player, and novice, all playing as Zerg against the Protoss-configured L.LM agent. Results, detailed in Ta- ble El show the LLM agent achieving competitive performance, par with Gold-level player. This

---

highlights the LLM's adaptability in strategic play and marks significant step towards integrating Al into competitive gaming environments on accessible home computing setups.
7 Analysis
7.1 Impact of Different Prompts
In evaluating the Chain of Summarization method using the GPT3.5-turbo-I1 model in TextStar- Craft II, we analyzed the effects of two different prompt types LLM agent performance when playing as Protoss against Zerg. The results, detailed in Table Al showed a notable improvement in performance with more complex prompts.
Table 4: LLM Agent Win Rates (%) vs. TextStarCraft II AI at Varied Difficulty Levels.
PROMPT
LV1
LV2
LV3
LV4
LV5
LV6
PROMPT 1
87.50
66.61 67
25.00
12.50
0.00
0.00
PROMPT 2
100.00
100.00
100.00
84.00
50.00
8.33
Simple Thought Chain: Using a basic prompt (see Prompt II), the LLM agent could perform elementary operations like worker production, base establishment, and basic combat unit production. However, this approach was limited in developing advanced strategies, such as research upgrades or comprehensive game analysis, indicating a narrower strategic depth with simpler prompts.
Complex Thought Chain: more intricate prompt (see Prompt D guided the LLM agent through a series of critical phases. including situation overview, analysis, strategic planning, and decision- making. This comprehensive approach enabled the agent to engage in advanced strategies like research upgrades, tech tree exploration, and complex military maneuvers. It proved particularly effective against higher difficulty levels (e.g., "Harder" Iv ), showcasing enhanced strategic capa- bilities.
Our analysis underscores the importance of complex prompts that replicate the thought processes of seasoned StarCraft II players. These advanced prompts are essential for LLMs to fully understand and strategically engage the sophisticated aspects of the game.
7.2 Policy Interpretability
Our analysis reveals a stark contrast in docision-makinne between AlphaStar and our LLM agent. While AlphaStar demonstrates superior micromanagement skills, it occasionally lacks rationality in its strategic choices. Conversely, the I.LM agent consistently exhibits logical decision-making, as evidenced by its proactive anticipation of threats and strategic planning, detailed in Appendix E and Appendix E
Phoze dalle
Steld Rasay
The Shrlf Nenry las noi becs conmmrie
Wie Gau
Ranage
Donce
(a) Alphastar failed to construct preemptive defen- sive structures such as Spore Crawlers to prevent Oracle incursions by the MasterPlayer, resulting in inadequate defensive capabilities against the Ora- cle harassment.
(b) The LLM agent demonstrated proactive defense measures by constructing Shield Batteries ahead of time, indicating its proactive awareness and prepared- for defensive strategies.
Figure 3:
The ability to construct defensive structures and anticipate dangers in advance: Alphastar (left) LLM agent (right).

---

Anticipating Threats: Figure @ illustrates a stark contrast between AlphaStar and the LLM agent in their ability to anticipate and respond to potential threats. In Figure aa, AlphaStar overlooks the impending danger posed by enemy Oracles, failing to recognize the need for adequate defensive structures. This oversight leaves AlphaStar's base vulnerable to a devastating attack, highlighting its inability to proactively assess and mitigate potential risks. Conversely, in Figure B.b, the LLM agent demonstrates remarkable foresight by analyzing the enemy's 's strategy and recommending ap- propriate defensive measures. The agent's s log data at 05:00 game time reveals its ability to infer the enemy's ' Roach-centre strategy based on the presence of key structures such as the Roach Warren and Spawning Pool. Armed with this insight, the LLM agent provides structured decision- framework to address the identified threat:
Enemy's ' Strategy: The enemy has established a Hatchery. Roach Warren, Extractor, and Spawning Pool. This indicates potential strategy centered around Roach pro- duction and early aggression.
Given the enemy's 's potential for early aggression with Roaches, we should prioritize defensive structures such as Photon Cannons and Shield Batteries. Additionally, con- sider scouting the enemy base to gather more information about their strategy.
Decisions:0: <BUILD PHOTONCANNON> <BUILD SHIELDBATTERY>
Flexible Unit Composition: AlphaStar often adheres to fixed unit compositions, leading to inef- fective strategies in dynamic scenarios (Figure [[3.-d). In contrast, the L.MM agent demonstrates adaptability in unit selection, effectively countering varied enemy tactics (Figure I..e-)). For in- stance, in Figure I the Zerg (Built-in AI) employs Hydra-Roach composition. The LLM agent, playing as Protoss in 09:00, recognizes the need to adapt its strategy based on the enemy's s ground- based army. The agent's thought process, as shown in the log, reveals its ability to analyze the situation and make informed decisions:
The enemy seems to be focusing on a ground- based army compo- sition, consisting of Roaches, Swarm hosts and Hydralisks.
Enemy's 's Strategy:
Key Information: The most important aspect at this moment is need to expand our unit composition and technology tree to counter the enemy's s strategy effectively. We should prioritize unlocking advanced units and upgrades to gain an advantage. Consider researching psionic storm at the Templar Archives to deal with the enemy s ground units effectively.
Decisions: 4: <RESEARCH PSISTORMTECH>
The LLM agent's superior performance in threat anticipation and unit composition adaptability stems from its structured, transparent decision-makin process. By analyzing the situation, iden- tifying key information, and making informed decisions based on the evolving game state, the LLM agent demonstrates a blend of human knowledge and logical reasoning. This approach enhances the interpretability of the agent's gameplay, facilitating better collaboration and strategic adaptation in complex scenarios, ultimately leading to more successful outcomes compared to AlphaStar's ' opaque strategies.
8 Discussion
TexiStarCraft enhances the use of LLMs for strategic decision-making in StarCraft II, showcasing their capabilities in adaptive strategy and crisis management. However, the frameworks reliance on rule-based scripts for micro policies, limitation to non-visual data, and a subset of the game's 's races may restrict the diversity and applicability of Al strategies. Additionally, resource limitations have constrained the performance capabilities our system. Despite these challenges. TextStarCraft Il establishes a new benchmark in RTS games, promoting deeper Al-human collaborative research. Future improvements will focus on integrating visual inputs, expanding race support, and optimizing resource usage to enhance strategic complexity and performance against established Al models.
10

---

Acknowledgements
This work supported by the National Science and Technology Major Project 2022/D00 16404.
This work was supported by the StarCraft AI and player communities. We extend our gratitude to Burny for providing detailed documentation and offering warm-hearted replies during the de- velopment process. are grateful to several StarCraft professional and amateur players who participated in our testing. Special thanks go to:
Zhiyao(aka. Hang (Champion of China StarCraft II College League, Profes- sional player of SSLT team)
ReT.ite (Champion of China StarCraft II College League, Professional player of SLT team)
Lolla (Champion of China StarCraft If College League)
sta.saaga (Professional player of SLT team)
firmameimmmntt +D (Champion of China StarCraft II College Tournament)
Their expertise and insights were invaluable to research.
References
[1] Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F,, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
[2] Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison. C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., et al. Dota with large scale deep reinforcement learning. arXiv 2019.
preprint arXiv:1912.0660
[3] Chen, L., Wang, L., Dong, H., Du, Y., Yan, J., Yang, F., Li, S., Zhao, P., Qin, S., Rajmohan, S., al. Introspective tips: Large language model for in-context decision making. arXiv preprint 2555 2023.
[4] Côté, M.-A., Kádár, A., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Hausknecht, M., El Asri, I.., Adada, M., al. Textworld: A learning environment for text-based games. In Computer Games: 7th Workshop, CGW 2018, Held in Conjunction with the 27th International Conference on Artificial Intelligence, IJCAI 2018, Stockholm, Sweden, July 13, 2018, Revised Selected Papers 7, pp. 41-75. Springer, 2019.
[5] Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang. D.-A., Zhu, Y., and Anandkumar, A. Minedojo: Building open-ended embodied agents with snternet-scale knowledge. Advances in Neural Information Processing Systems, 35: 35:18343-18362, 2022.
[6] Gong, R., Huang. Q. Ma. X., Vo, H., Durante, Z.. Noda, Y.. Zheng, Z., Zhu, S.-C., Ter- zopoulos, D., Fei-Fei, L., et al. Mindagent: Emergent gaming interaction. arXiv preprint arXiv:2309.0999 09977, 2023.
[7] Han, L., Xiong, J., Sun, P., Sun, X., Fang. M., Guo, Q., Chen, Q., Shi, T., Yu, H., Wu, X., et al. Tstarbot-x: An open-sourced and comprehensive study for efficient league training in starcraft ii full game. arXiv preprint urXiv:2011.137 137299 2020.
[8] Hong, S., Zheng, X., Chen, J., Cheng, Y., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z.,, Zhou, I.., Ran, C., et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint 2rXIv::308.000352 2023.
https://aiarena..m
https:
E7 27%A9\BA

---

[9] Huang, R., Wu, X., Yu, H., Fan, Z., Fu, H., FU, Q., and Wei, Y. A robust and opponent apponent-aware aware league training method for starcraft ii. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[10] Jin, X., Wang, Z., Du, Y., Fang, M., Zhang. H., and Wang, J. Learning discuss strategically: A case study on one night ultimate werewolf. arXiv preprint arXiv:2405. 19946, 2024.
[11] Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., and Ghanem, B. Camel: Commu- nicative agents for" mind" exploration of large scale language model society, arXiv preprint 27X1v:2303.17760, 2023.
[12] Liu, R.-Z., Guo, H., Ji, X., Yu, Y., Pang, Z.-J., Xiao, Z., Wu, and Lu, T. Efficient reinforce- ment learning for starcraft by abstract forward models and transfer learning. IEEE Transactions on Games, 14(2):294-301 2021.
[13] Liu, R.-Z., Wang. W. Shen, Y., Li, Z., Yu, Y., and Lu, T. An introduction miin-alphasttrr arXiv preprint arXiv:2104.06890, 2021.
[14] Liu, R.-Z., Pang. Z.-J., Meng, Z.-Y., Wang, W., Yu, Y., and Lu, On efficient reinforcement learning for full-length game of starcraft ii Journal of Artificial Intelligence Research, 75: 213-260, 2022.
[15] Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. Agentbench: Evaluating ilms as agents. arXiv preprint arXiv:2308. 03688, 2023.
[16] Mathieu, M., Ozair, S., Srinivasan, S., Gulcehre, C., Zhang, S., Jiang, R., Le Paine, T., Zolna, K., Powell, R., Schrittwieser, J., et al. Starcraft ii unplugged: Large scale offline reinforcement learning. In Deep RL Workshop NeurlPS 2021, 2021.
[17] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 11227330-27744 2022.
S., Slama, K., Ray, A.,
[18] Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, S. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442 2023.
[19] Peng, P., Wen, Y. Yang. Y., Yuan, Q., Tang, Z., Long, H., and Wang, J. Multiagent nets: Emergence of human-level coordination in learning to play starcraft combat games. arXiv preprint arXiv: 1103. 10069, 2017.
andirectionally-cordmmm
[20] Shridhar, M., Yuan, X., Côté, M.-A,, Bisk, Y. Trischler. A., and Hausknecht, M. Alf- world: Aligning text and embodied environments for interactive learning. arXiv preprint 2.Xiv20010033666 2020.
[21] Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312 1805, 2023.
[22] Touvron, H., Martin, I.,, Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint
2023.
[23] Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, S., Yeo, M., Makhzani, A.. Küttler, H., Agapiou, J., Schrittwieser, J., et al. Starcraft ii: A new challenge for reinforcement learning. arXiv preprint arXiv:1708s 04782, 2017.
[24] Vinyals, O., Babuschkin, I., Czarnecki, M.M Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782)3350-3333 2019.
[25] Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open-ended embodied agent with large language models. arXiv preprint drXiv:2305.16211, 2023. 2023.
12

---

[26] Wang, X., Song. J., Qi, P., Peng, P., Tang. Z.. Zhang, W., Li. W., Pi, X., He, J., Gao, C., et al. Scc: An efficient deep reinforcement learning agent mastering the game of starcraft ii. In International conference machine learning, pp. 10905-10915. PMLR, 2021.
[27] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F, Chi, E., Le. Q. V., Zhou, D., al. tf prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24565 2022.
[28] Wu, Y., Min, S. Y., Prabhumoye, S., Bisk, Y., Salakhutdinov, R., Azaria, A., Mitchell, T., and Li Y. Spring: Gpt-4 out-performs rl algorithms by studying papers and reasoning. arXiv preprint arXiv:2305. 15466, 2023.
[29] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao. Y. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
[30] Zeng, A., Liu, X., Du, Z.,, Wang. Z., Lai, H., Ding, M., Yang, 7., Xu, Y., Zheng, W., Xia, X., Tam, W. L.,, Ma, Z., Xue, Y., Zhai, J., Chen, W., Zhang, P., Dong, Y., and Tang, J. Glm- Gim--30b: An open bilingual pre-trained model, 2022.
[31] Zeng, Y., Liu, G., Ma. W., Yang, N., Zhang, H., and Wang, J. Token-level direct preference optimization. arXiv preprint arXiv:2404. 11999, 2024.
[32] Zhang, C., Yang, K., Hu, S., Wang, Z., Li, G., Sun, Y., Zhang, C., Zhang, Z., Liu, A., Zhu, S.-C., et al. Proagent: Building proactive cooperative ai with large language models. arXiv preprint arXiv:2308. 1339, 2023.
[33] Zhang, C., He, Q, Zhou, Y., Liu, E. S., Wang. H., Zhao J., and Wang, Y. Advancing DRL agents in commercial fighting games: Training, integration, and agent-human align- ment. In Forty-first International Conference Machine Learning, 2024. URL https: //openreview.o htt.:/ewwweeeoooor//eeeeettttt
[34] Zhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., Wang, X., al. Ghost in the minecraft: Generally capable agents for open- world enviroments via large language models with based knowledge and memory. arXiv preprint arXiv:2305.17144 2023.
13

---

Impact Statements
This work introduces novel method for addressing real-time strategy (RTS) tasks and strategic missions using Large Language Models (LLMs), expanding their application beyond traditional domains. Our approach enhances the interpretability of decisions made by LLMs, offering insights into the language policies that govern Al behavior in complex scenarios. These advancements have potential implications for sectors requiring nuanced decision-making and strategic planning, further bridging the gap between Al capabilities and human-like reasoning in dynamic environments.
Broader Impact
This paper addresses both the positive and negative societal impacts of employing Large Language Models (LLMs) for real-time strategy gaming environments, such StarCraft II. On the positive side, enhancing interpretability through language-driven strategies fosters greater understanding and transparency in Al decision-making, which be beneficial across various sectors that require plex strategic thinking. However, the potential negative impacts include the risk that Al-generated strategies recommendations might be misleading. This could adversely affect human decision- making, particularly in scenarios where users rely heavily on AI suggestions without sufficient over- sight or understanding. Such misleading advice could lead to suboptimal even harmful decisions in critical areas like security or management. To mitigate these issues, we advocate for rigorous validation of AI advice, comprehensive user education on the AI system's capabilities and limita- tions, and the development of fail-safes that can alert users to potential inaccuracies or biases in Al-generated strategies.
Code Of Ethics
This research fully adheres to the NeurIPS Code of Ethics. We have ensured that all experiments, data handling, and dissemination of results maintain high ethical standards and transparency. The methodologies and technologies used in the study applied in controlled environment with no real-world deployment, mitigating the risk of harm. Additionally, we have preserved the anonymity of any data used during the training and testing phases comply with privacy and ethical guidelines.
14

---

A Experiment Setup and Metrics Explanation
A.1 Experiment Setup
Agent and Opponent Selection: To ensure a consistent and controlled experimental environment, LLM agents are configured play Protoss against Zerg AI opponents. This setup allows for systematic evaluation of strategic capabilities across a range of difficulties. The difficulty levels are defined as follows to cover comprehensive spectrum of challenges:
Level
3
4 Hard
5
BLZ Difficulty
Very Easy
Easy
Medium
Harder
Level
6
7
8
9
10
BLZ Difficulty
Very Hard
Elite
Cheat Vision
Cheat Money
Cheat Insane
Table 5: Al Difficulty Levels for StarCraft II
Map Selection: For experiments, selected the Altitude LE and Ancient Cistern LE maps from the 2023 StarCraft esports Iv1 ladder. These maps provide a variety of strategic challenges that are representative of current competitive play. Additional details about these maps can be found
on their respective Liquipedia pages: https://iqquipedia htt.:/uuupedaaaeeetteaeaea pttp:ett/tttttee htt.://aaarrraaf//ttituuuuua
LE
and https://Liquipediaa helrlannetttttttteeee wtt.s/trrrrafft/iiieeeiita Cinteen__sstee_ee LE
Parameter Settings: The temperature parameter is fixed 0.1 to focus on strategy-driven actions over randomness.
Game Version:
Our experiments were conducted across three different versions of the game to
ensure robustness and applicability of the results. The versions tested include Patch 5.0.11, Patch 5.0. and Patch 5.0.13.
A.2 Evaluation Metrics:
Building on StarCraft II's established player performance analytics, our evaluation framework in TextStarCraft II integrates these insights with custom adaptations comprehensively assess LLM agents gameplay strategies.
The most crucial metric, Win Rate directly reflects the agent's capability and perfor- mance within the environment. It is calculated as the percentage of games won by the agent out of the total games played.
Win Rate:
Population Block Ratio (PBR): PBR is measure of the agents macro-management effectiveness. particularly in resource allocation and population growth. It is defined as:
PBR Time at Population Cap
Game Duration
Here, the metric caleulates the ratio of time spent population capacity to the total game time until the agent first reaches maximum supply (200/200). A higher PBR indicates poorer macro-strategic decision-making and ineffective planning.
Resource Utilization Ratio (RUR): RUR evaluates the agent's 's efficiency in resource management throughout the game. It is calculated as:
RUR Total Minerals + Total Gas Used
Game Duration
This metric assesses the total resources used against the duration of the game until the agent first reaches maximum supply. A higher RUR signifies underutilization of resources, reflecting weaker macro-strategic capabilities.
Average Population Utilization (APU): APU measures the efficiency in utilizing the available population capacity. is calculated as:
Used Population 7th step
APU
Population Cap at ith step
15

---

The metric averages the ratio of used population to population cap until the agent first reaches maximum supply. higher APU suggests better utilization of population and more effective macro- management.
Technology Rate (TR): TR assesses the extent which the agent explores and utilizes the technol- ogy tree. It is defined as:
Completed Technologies
TR Total Technologies Available
(
This metric calculates the proportion of completed technologies and buildings to the total available, from the start the end of the game. TR indicates the agents tendency towards technological advancement and is not necessarily indicative of better or worse performance.
Data Collection and Analysis: Due technical constraints with the sc2reader library for Al- generated replays, we utilized gameplay log data for in-depth analysis. This approach ensures a thorough and precise evaluation based on tailored metries.
A.3 Compute Resource
To fine-tune open-source Large Language Models, we utilized two NVIDIA A100 40GB GPUs, providing the necessary computational power for extensive model training. The entire experiment took approximately 70 hours.
Training Phase:
Testing Phase: The development and testing of these fine-tuned models were performed using two NVIDIA A100 40GB GPUs. For running the StarCraft II environment, we employed an NVIDIA GeForce RTX 3060 GPU paired with a 13th Gen Intel(R) Core(TM) 15-134000 processor operating 2.50 GHz, ensuring smooth execution the game simulations.
A.4 Human Experts
In the course of our research, we engaged 30 StarCraft II masters and grandmasters, including professional and semi-professional players, to participate in our experiments. To safeguard their privacy, we have chosen not to disclose their personal information in this paper.
Each participating player was compensated with 15 USD for their involvement in the study.
16

---

B Experiment Figure
Probe
Seisor Ulitt
Protoes
Ohoarver
basc
Eoeny
0.2 010
Eodny stan
locati
Phosss Hirs location
(a) early game stage, scouting is crucial. The Protoss have decided to dispatch a Probe to scout the enemy's main base.
(b) The LLM agent values enemy scouting. The Protoss Observer spotted Brood Lords and advanced tech in the Zerg's 's third base.
Diann bas ooar
(c) The Protoss employs Shield Batteries as a pivotal defensive strategy against Zerg aggression.
(d) The Protoss are utilizing the Chronoboost ability to accelerate the "Protoss Air Weapon Level 1" up- grade in the Cybernetics Core.
Figure 4: Early and mid- game snapshots of the Protoss (LLM agent), emphasizing defense and scouting.
Protoss Unit Preferences: 2015, 2020 2023 Premier Teumaments Compared with LLM Agent build-in in PvZ
Premier Toumsments
Toumamants Agent
Frequency
cc
62
0.1
Zealst
Sentry
Slatker
Adeptt
High Tamptar
Dark&Templa
Archon
immorial
Colosaus
Disruptor
Ohserser
Warrp .m
Pievaria
VeldRaay
Oracie
Tempees
Cerreer
Mathership
Protoss Units
Figure 5: Comparison of Unit Preferences: Analysis of LLM Agent in 30 Games Professional Players in Esports Tournaments from 2015, 2020, and 2023
17

---

PHALK DOW
A
3 18
(a) The Protoss strategically transitions to Void Ray-centric composition, adept countering the might of Zerg's Ultralisks.
(b) Protoss defended against Zerg's third wave with phoenixes, tempests, and voidrays.
015 0100
18 29 28
Duspos 20
(c) The Protoss, employing the classic combination of Colossus and Voidray units, successfully deci- mated the Zerg's 's expansion base.
(d) The LLM agent uses stalkers, zealots and ob- counter the Harder Al's attacks, gradu- ally shifting towards a powerful unit and Psionic Storm strategy.
Figure 6: Mid to late-game visuals of the Protoss (LLM agent), showcasing mixed-unit assembly and troop transitioning.
Table 6: Statistical Evaluation of Models
Mean Score
Std Dev
T-test
Levene
Pearson Corr
p-value
p-value
Model
GPT-4
Human
GPT-4
Human
Bard
3.70
3.79
1.02
1.12
0.00
0.4078
0.4718
ChatGPT
5.00
4.42
0.83
0.97
0.00
0.0000
0.0001
Claude2
4.69
3.91
0.70
1.12
-0.00
0.0000
0.0000
GPT4
5.17
4.77
0.71
0.93
-0.00
0.0000
0.0001
18

---

REASLA
Zan arasy
Protoss combat Zarg bot merely lass
amly attak
5. 88
20 88
(a) Amidst rising tensions, the Zerg initiates an as- sault on the Protoss base.
(b) The Zerg's roach and hydra assault gains momen- tum against the Protoss.
Zong-eng
base Pootoss
vildky combat
Zera
(c) The Zerg's onslaught continues, pressuring the Protoss's 's second base.
(d) The Zerg attacking the Protoss main base, and the Protoss are defending with shield batteries voidrays.
Ingoosessi
Protess anmy mondly diminete Zing unmty
Protess baset
08
2 00.39
(c) Protoss defended with voidrays and phoenixes, leaving roach damaging their weakened base.
(f) Protoss prioritizes cconomy and voidray produc- tion, alongside rescarching phoenix range upgrade.
Pretisk army merely purge Zrrg amy
Zorp Prctucs
(g) Protoss defends Zerg's 's roach and hydra attacks with voidrays and phoenixes.
(h) Protoss protected their third base from Zerg roach hydra assault with voidrays and phoenixes.
Figure 7: Protoss Strategic Defense Against Zerg Assaults: the depicted scenes from StarCraft II, the LLM agent, controlling the Protoss, initially faces losses including the destruction of the second base during Zerg Roach-Hydra assault. Despite this, the Protoss strategically expand their econ- omy and transition to an air force with Void Rays and Phoenixes, successfully repelling subsequent Zerg attack.
19

---

C Latency and Real-Time Feedback
The latency in our system is primarily determined by the inference speed and model size of the LLM. By leveraging fine-tuned open-source LLMs, our approach can achieve real-time feedback. Our Chain of Summarization (CoS) method crucial enabling this real-time feedback, significantly outperforming traditional CoT methods in terms of responsiveness.
For instance, with CoS, even smaller LLMs like the fine-tuned Qwen-2 2 7B achieve real-time in- teraction (100 ms/step) and demonstrate competitive performance against human players. This level of real-time capability is not possible with standard CoT methods, highlighting the effectiveness of our CoS approach in facilitating swift strategic decision- makinn in complex, dynamic environments.
Although larger models such as GPT-4 Claude, and Gemini still face challenges in achieving real- time feedback (Is/step) due to their size, complexity, and network transmission factors, our CoS method significantly reduces their response times compared to CoT. This improvement allows for near-real-time performance in many scenarios. We are continuously optimizing our system to further enhance the real-time capabilities of these larger models.
Below are the delays observed for each step when different LLMs interact with the CoS method:
Model Type
Model Name
Delay (each step)
Finetuned Spurce LLM
Qwen2-7b
98ms
Finetuned Open-Source LLM
Qwen2-1.8b
64ms
Finetuned Open-Source LLM
LLAMA2-7B
102ms
Closed-LLMM
GPT3.5
1.2s
Closed-LLM,
GPT4
2.3s
Closed-LLM.
Gemini-PRO
0.3s
D Prompt and data Example
Figure & IO. III and depict sample data from the interaction process between the Chain of Summarization method and the TextStarCraft II environment.
There two types of system prompts in the A section.
1. Promptl: You an AI trained in analyzing and summarizing StarCraft II games. You understand the nuances and strategies of the race race. Based on the summaries of multiple rounds in game, want you analyze the game progression in structured way. Your analysis should include the following aspects:
(a) Information Overview: Provide brief overview of the current situation based on all the rounds.
(b) Current Game Stage: Determine the stage of the game based on the information of all rounds. Is it the early game, mid-game, or late game?
(c) Our Current Strategy:From the information of all rounds, infer what our strategy might be.
(d) Enemy's Strategy:Infe what the enemy's strategy might be, based on the informa- tion available.
(c) Key Information: significantly influenced the game.
Highlight the most important aspects from all rounds that have
2. Prompt2: You are an Al trained analyzing and summarizing StarCraft II games. You understand the nuances and strategies of the self.race race. Based on the summaries of multiple rounds in a game, we want you to analyze the game progression in a structured way. Your analysis should include the following aspects:
(a) Game Overview:Provide a brief overview of the current situation based on all the rounds.
(b) Current Detrmine the stage of the game based on the information of all rounds. Is it the carly game, mid-game, or late game?
20

---

observation
Game time
enemy UaltDypeld.DVVESEEER
dark shaine count:
time: 15:27
enemy. UnitTypeld.OV ERL 3
rbboiics, bey count:
enemy. UnitTypeld.7ERGH Z
Neet beacon count:
Resource information
enemy Satillyyld.QUEBBR
plamning, pylon, count:
worker supply: 47
enemy UnitTypeld.HY DRA BISKBUROOWED
planning ges buildings count:
mineral:
plamning gateway count:
4872
Building Information
plaming forge, count: 0
supply left:
pylon, count: 7
planning photon, camnon count:
supply cap: 131
ges buildings, count: 10
planning, shield bettery count: plamingg wasp gate count: 0 planning cybemetics
supply used: 102
gateway_ count 0
base count:
forge
count:
planning bese count:
photon cannon count: shield, battery count: warp gate count: 5
planning twilight, council count: planning robotics facility count: plaming statgata count I
Millttrry Information
army. supply: 45
ey gybermetics core, count: twilight, council roboiics facility count:
planning templar archives count planning dark shrine count:
eaemy units 33
enemy UnatllypeldINNESSTTI INFESTOR: 2
planning robobics bey count planning fleee beacon, count:
enemy Unittyppel..R00 ROA CH: 16
stalgate count:
cnemy UnitTypelé.HY DRALISK:
templar archives count: 0
Unit information
planning worker count:
6. Reswarch information
Zualot, count: 0
planning Zealot, count: 0 planning stalker count 0 planning sentry
warpgate research, status: protoss amoor level research protoss air armor level 2 research status: 0 protoss aic emmor level 3 research status:
stalker count: 0
sentry
adept count: 0
planning adept countz 0 planning high. templar count: planning dark templar count: 0 planning. immortal count: planning colossus count: planning, disruptor count: planning archon
high. templar count:
protoss, ground armor Jevel research status: 0 protoss, ground armor level research status: 0
dark templar count:
immortal count:
protoss, ground armor level reseerch status: 0 protoss ground weepon level 1 reseerch status: protoss ground wespon
colossus count:
disruptor count: 0
ressurch status:
arohon count:
0.0
ground weapon level ressarch. statas: protoss shield level researrch status protoss shield level research status: protoss shield level research. status:
observer count:
planning observer count: 0 planning warp prism, count: planning phoenix count: planniny, voidmy count: planning Oracle
warp prism, count:
phoenix count:
vuidmy count: 8
Oracle
7. Enemy Building information enemy UsltTYpedd.HATCHEEEE enemy UnitTypeld FX TRACTOR: enemy UnitTypeld NINGPOOL:
Carrier count:
planning. Cartier count:
tempest count:
planning tempest
mothership count: 0
planning mothership count: 0
Ememy Unit information
enemy
Unstlypeld.NNEESTOR INFESTOR:
enemy
UnitTypeld CH: 16
enemy
UnitTypeld.HH DRALISK:
UnitTypeld.OV EVRREER--OEEEER---22
enemy
Soaullyaad.VVELLLDTRR..
enemy
Unitfypeló ZERGLING:
eaemy
Unitlypelå.QU QUE.lyyppdd.UEEN::::
enemy
Unitlypeld.HY BRALISSBURROWED:
Figure 8: An example of the TextStarCraft II's Raw text observation.
(c) Our Situation:
Describe our current status in terms of:
i. Units and Buildings:Analyze the state of our units and buildings.
ii Economy:Evaluate economic condition, including resource collection and us- age.
iii. Technology :esccribb the status of our technological research and what technolo- gies we have unlocked so far. Analyze our technology tree, indicating the available and potential upgrades or units.
(d) Our Strategy:Infer our potential strategy based on our current situation and the infor- mation of all rounds.
(c) Enemy's Strategy:Infer the enemy's 's potential strategy, based on the available infor- mation.
(f) Key information:Highligee the most important aspects from all rounds that have sig- nificantly influenced the game.
(g) specific prompt
i. Zerg :For Zerg, pay attention to whether there are enough larvae. If not, we should consider adding the INJECTLARVA command to the queue.
21

---

L1 results
At 15:27 game time, our curment StarCraft situation follows:
rybemesics core, count,
In-procese
twilight, council, count:
planning count:
robotics, facility, count: 1
planning statgale count:
Resources:
statgatee count:
planning voidnay count:
gume time: 15:27
planning base count:
worker supoly:
planning statgate count:
Research:
mineral: 3005
warpgate research status:
gas 4872
protoss air armor Jevel, research status:
supply left:
Units:
supply cap: 131
enemy units count: 32
Enemy Information:
supply used:
base count:
enemy
units count: 32
ammy supply: 41
pylon count:
enemy
UnitTypeld
enemy units count: 32
gus buildingss count: 10
unemy
UnilTypefd. ERLORDTRA NSPORT:
count: 6
shield battery count:
enemy
UnitTypeld.HY DRALISKBURROWS
warp gate count: 5
enemy.
UnttTyped,ZERGGL..N
Builddigg:
cybemetics
enemy
UnitTypeld.OV ERSBER:
enemy units count: 32
twilight, council, count: 1 robotics facility count: statgate count
enemy
UnifTypeld.HY DRALISK:: 5
base count: 6
enemy. Unirdypeldd UniiTypef,.NFFESTOR
pylon, count: 7
enemy UniiTypeld.QUEEN
guu huildings count: 10
observer count
enemy UniiTYypf&,.AATCHEEE
shield battery count
voidaa count:
enemy UniiTYyptd.EXTRAACTIII
worp_gott, count: 5
planning voidray count: 3
unemy Www.TTyyoo....AWWWNGGGG.GGG
Figure 9: LI summarization
tystem grompt
You Al trained in analy mailyzing and summarizing StarCraft 1l games. You understand the nuances and strategies of the protoss for zerg) race. you to analyze the game progressoon in structured way. Your analy sis should Inc Based on the summartes of multiple rounds in game, we you
lode the following aspacts:
rounds. 2. Cument Game Stage: Deternine the stage of the game based on the information of all rounds. Is it the early game, mid-game, or late game?
Overview: Provide hrief overview of the current situation based
3. Our Situation: Describe our current status in terms of:
Units and Buildings: nalyze state of our units and buildings.
Ecanomy: Evaluate our economic condition, including resource collection and usage.
3.3 Technology: Descrbbe the status of our technological research and what technologies we have unlocked far Analyze our technology tree,
Indicating the available and potential upgrades or units.
Our Strategy: Infer our potential strategy hased on our current situation the information all rounds.
Enemy's Strategy: Infer the enemy's potential strategy, based the available information.
6. Key Information: Highlight the most important asgects from all roudss that have significantly influenced the game.
For Protoss, keep an eye on Nerus's energy to Chrono Boost important structures.
Based the game situation and strategies used both sides, provide specific suggestions for the following
1.0ur Strutegy: Propose adj ustments to current strutegy to countar the enemy's 's moves and capitalize
strengths. Units and B bul dings: Offer ways to enhance our unit compositoo and improve our building lay ayoout suited to the current stage of the game.
3. Economy:: Recommend better practices for resource gathering and usage, in line with our strategic needs.
current research status and technology tree. Lastl, consider the current situation and the suggestions provided, make (K) actionable and specifie decisions from the action dictionary protoss action, dict. This dictionary comprisss four categories of actions: unit productonn. building construction, technology research, and other actio ns. Remember align these decissozs with the current stage of the game, and avoid proposing actions that are not currenly feasible.
Technolooy:: Suggest focused research paths to guin technological adv antages, considering
Figure 10: System prompt from L2 summrization. This prompt enables the Large Language Model to engage in playing StarCraft 2. The process includes several key steps: Situation Overview, Sit- uation Analysis, Strategic Planning, Opponent Strategy Analysis, and Strategic Recommendations, concluding with the Decision-Making Process.
22

---

LLM Analysis
Overview:
15:20 game time, situatio is as follows 49 workers, 2970 minerals, 4884 gas. Our supply cap at 131 with supply left, and our amm supply is 41. We have six bases and decent number of buildings and units.
Current Game Stage:
B Basad game time and resourbe availabalit,, still mid- sood--gamee stage.
3. Our Situation:
Units and Buildings: We have diverse unit composition, including Voidrays, a deceat number of buildings, including a Cy Cybemetics Core,
Twilight Council, and Robotics Facility. We also have guod number of Pylons und gus buigs
3.2 Economy: Our economy is stable, with sufficient number of workers and good balance of minerals and gus.
3.3 Technology: We have started reseorching W Warpgate and Protoss A An mor Level 1. However, there is room for Fürther technological development and reseurch.
4. Our Strategy:
Our strutegy soums to be focused on air dominance with Voidrays. We should continue leveruge our air superiority while expanding our tachnological capabilitie.
5. Ememy's 's Strategy:
The enemy seems to be focused on ground based army composition, with Roaches, Zerglings, and Hydralisks. They also have some support units Overseers and Infestors. We should cautious potential ground attociss and plan defenses accordingly.
6. Key Information:
The must important uspect at this mumunt is our air superiority with Voidmys, Howuver we need to be prepared for potumtial ground- bosed attacks from enemy's 's Roaches drydklisks, future success will depend on we can mointain our dominance while countering the enemy's ground forces.
Figure 11: LLM analysis: LLM take Situation Overview, Situation Analysis, Strategic Planning, Opponent Strategy Analysis these four stages.
ii Protoss:For Protoss, keep an eye on Nexus's energy to Chrono Boost important structures.
(h) Based on the game situation and strategies used by both sides, provide specific sug- gestions for the following areas:
Our Strategy: moves and capitalize on our strengths.
adjustments to current strategy counter the enemy's
i. Units and Buifdings:Offer ways to enhance our unit composition and improve our building layout, suited to the current stage of the game. iil Economy::ecomm Recommend better practices for resource gathering and usage, in line with our strategic needs.
iv. Technology:Sugee :uggestt focused research paths gain technological advantages, con- sidering our current research status and technology tree.
(i Lastly, consider the current situation and the suggestions provided, make K actionable and specific decisions from the action dictionary. This dictionary comprises four cat- egories of actions: unit production, building construction, technology research, and other actions. Remember to align these decisions with the current stage of the game, and avoid proposing actions that are not currently feasible.
23

---

LLM Suggestions Decisions
Suggestions:
Our Strategy:
We should continue to producu Voidray's und maintaain air superiority. However, we should alsu focus un developing strung ground ammy to counter the eaemy''s Roaches and Hy dradisisss.
Units and Buildings
Consider building more G Gateways and producing units Like Stalkers and Immoctals to strungthen our ground amy A Aditionnlly, expanding our shield battery count can provide better defense equinst ground attockss
Economy:
Ensure that our workers are efficiently gothering resuuroes. Consider exponding to new bases to further boost our ecunomy and support our unit pouduction.
Technology:
Continue reseurching upgrades for our air units, such Protoss Air Amor Level and Level A Additionally, consider reseurching upgrades for ground units, such as Protoss Ground Weepons and A Amor.
Decisions:
0: <TRAIN STALKER>
IN IMMORTAL>
<BUILD CATEWAY>
<BUTLD SHIEL DBATTERY
4: NA BASE>
Figure 12: LLM suggestions and decision: LLM give useful Strategic Recommendations and finally take the Decision- Decision-Making Process. Our The actions of this infer train stalker, train immortal, build gateway, build shield battery and expand base.
can extract the actions from Decisions.
24

---

E Alphastar and Ilm agent
Joy Shaie Dueen
Giane 24.52: Libertto-heerry anmy establishes Rones, Zorg (AlghaStar) persists Queens which ineffeetive against the Liherators.
(b) Protoss player) employs Shield Battery, mortal strategy. response, Zerg (AlphaStus) Spore for defeuse, which is ineffective against the current Protoss strutogy.
Protess (Master fiolds Immertal and Disrupice
Terran (Master player) empploys bio tank composition (Marines. Marauders, Tinks) counters with a large number of Cormuptors, which are ineffective against Teran's ground-facused cound-focuseddeee army.
(d)
army. Despite strong cccnomy, Zorg (AlphaStrr) comtinues producing Roaches, a unit countered by the Protoss forces, without opting for tran- sition.
Zeeg (AlphaStur)
proccct
Qunea
Dl
(Bailt-in utiizes Hydrr--Rochh army compositon,
(Built-in employs Hydra Roach compasitinn countered
effectively by Procoss Agent) who defends with Voudrays and continues produce more, capitalizing on their effectiveness spainst the Hydra Roach combination.
by Agent) using Stalkers detense, while also researching Priunic Stoem technology effectively manage the Zerg foroes.
Tokant gooid prag
Guraa 1X
Zorg
Uttalisk-heawy army,
(h) Protoss (LLM agent) ouly possesses counter op-
(LLM Agent) responds with a composition of Stalkers and Immartals, ze- tively producing more Immostals, highly effective against Ultraliks..
ponent's but also can expand its diverse aary of strategies and amy com- positions, such air
Figare 3:: The ahility to whapt strategies hased on the composition of enemy foncos: AlyhaStner (a, b, c. d) vs. 1.5 Agent (e. h.)).
25

---

F Policy interpretability examples
In Section 6.3, we introduced the concept of policy interpretability within our LLM agent. We now present four illustrative examples to demonstrate how the LLM agent can generate reasonable strategies and decisions in TextStarCraft Each example comprises three components:
LI Results Figure: This displays the input of the multi-frame summarization method.
L.LM Output: The output of the multi-frame summarization method, which includes the
LLM's analysis, suggestions, and decisions.
Decision Figure:
Illustrates the actual implementation within the StarCraft II game engine.
F.1
Example
results
00:4 time, StarCaft
034 game time, current SterCraft sruation followes
At CD:4G game time, our current StarCroft ahuation is as follows
sitabon followe
Resouroes
Rescuroes:
Resources:
game, time: CD:66
-game, time: 00
gamee, Jime 00.46
worker supply: 15
-wokker supply 15
worker supply 15
minerrl: 105
-mmerat 126
miheeat :066
supply Jeft:
-susp-yy, left:
sugply. left 7
Hupply zope 23
-susp-yy. 25
-urpely caps
supplyy 15
suudttyuued
suuplyy used:
bacee pouut
base
Ouildings
Duilding::
Buildings:
count::
been, count:
base
piioon, dount:
pyfoo coun
gas. www.tdinggsccommm count:
planning ças buildings.cc count
planning g. gas buildings,
2
planning.worke
planningg
planning wow.orker
Units
base, couutt
Saeat 1
soytn_ccoaa couun:
buildings, count:
Gat, www.iiiliigg...o Loount:
ppanning ces, buildings
planning buildings, count:
-llamiingg gar, bulicings, count:
planning you folkee counth
plarming werker count:
plbininngg worker count
proces::
In process:
In-proosee
planning ças, buildinge sulldine.couu::2
planning gas, buildings, count planning worker, pountt
pdlariing.s ge huilcings, pount:
planning worker 1
plbining..ww www.eng.wwooooooooooooooo pwokker.count sount
Enemy informaiore
Enemy Information
Evemy Informaiion
No keown enemy urits
No known enemy units
enown enemy un
Figure 14: LI results of 00:46 period
LLM output:
1. Game Overview: At 00:46 game time, our current situation is as follows. We have 15 workers and 105 minerals. Our supply cap at 23 with 7 supply left, and our army supply is 16. We have one base, one pylon, and two gas buildings. There are known enemy units.
2. Current Game Stage: Based on the limited resources and small army size, we are still in the early game stage.
3. Our Situation:
- 3.1 Units and Buildings: We have a small worker force and limited infrastructure. We have one base, one pylon, and two gas buildings. There are units present except for the workers.
3.2 Economy: Our economy is just starting, with a small number of workers and limited resources.
3.3 Technology: There is no evidence of any technological development or research at this stage.
4. Our Strategy: At this early stage, our strategy should focus on expanding our econ- omy, establishing more bases, and increasing our worker count. We should also consider scouting to gather information about the enemy's strategy.
26

---

5. Enemy's Strategy: Since there is no information about enemy units, it is difficult to determine their strategy. However, important remain vigilant and be prepared for any potential attacks.
6. Key Information:
The most important aspect at this moment to quickly expand our economy and increase worker count. This will provide with stronger economic foundation and allow us to produce more units and buildings.
Suggestions:
Prioritize expanding our economy by building additional bases and in- creasing our worker count. Consider scouting gather information about the enemy's strategy.
1. Our Strategy:
2. Units and Buildings: Focus on building additional pylons to increase our supply cap and support future unit production. Consider constructing additional structures such as gateways or a robotics facility to start producing units.
3. Economy: Allocate workers to gather minerals and gas efficiently. Aim to saturate our current base and expand to new resource locations soon possible.
4. Technology: At this early stage, is not necessary to focus technology research. However, consider building a cybernetics core to unlock more advanced units and upgrades in the future.
Decisions:
0: <BUILD NEXUS>
1: <BUILD PYLON>
2: <BUILD GATEWAY>
3: <TRAIN PROBE>
4: <SCOUTING PROBE>
PRODUCTION
The Protoss are ourvently employing Probes for scouting purposes.
Protoss initial start
Zerg initiaah
A.I. (Hard)
20 .000
15
PRODE
start
96
0
vaseuigg
15
Jen
Figure 15: Decision of LLM Agent: Scouting probe
27

---

F.2
Example 2
ros ats
StarCroft
StarCren Craftil
A: 03:4 folllows.
StarCroftt
situtiio
sitastion follows
Resources:
Resources supppyy
Resources: 03:44
05:43 350
-eupplyy_cap
minerrl 305
suupply_usodd
S06
let:
cap:
Suildinge
bose,
base,
Pylon_.currt
build
builldingge.oo countt-
process:
gallwway
gaonway
paaninng galeway
oycermetis zooe
planning
planning gateway, count: planning
Research
planning Zealot
19r gigate research 088092299250222566
grosees
Eneryy wWw.://EEEPPPHHHHTHHHR...
gateway
weriggtee
cenenny www.y/nnnnyyyeeeeeeeeeeeoee
eenertyy ypedd WWWp:///0SS////NGOOOOOOO
Eremy Intormatteeoo
enemy. UAt.. enemy. Uewpg.Unttlyeeeeeecc0oe EXT.PTTTPEE..T www.y.unttypeettSP/WWWPPPPP
enemy Unttypeld HATCHERY.
Figure 16: LI results of 03:43 period
LLM output:
1. Game Overview: At 03:44 game time, our situation relatively early in the game. We have 24 workers, 305 minerals, and 310 gas. Our supply cap is 54 with 23 supply left, and our army supply is 6. We currently have two bases.
2. Current Game Stage: Based the resource levels and army size, are still in the early game.
3. Our Situation:
3.1 Units and Buildings: We have limited number of units, with only 1 Zcalot in our army. Our building count includes 3 pylons, 4 gas buildings, 2 gateways, and I Cybernetics Core. Additionally, we are planning to build gateway and Zealots.
3.2 Economy: Our economy is modest, with small worker supply and resource pool. It's crucial to boost our worker count and resource collection rate.
3.3 Technology: We have started researching Warpgate technology, which is positive step. However, our lack of advanced units indicates that our tech tree has not progressed significantly.
4. Our Strategy: Our current strategy appears to be a cautious early-game approach, focus- ing tech development through Warpgate research.
The enemy seems to be expanding their Zerg infrastructure with a Hatchery, Extractor, and Spawning Pool. They might be preparing for Zergling production.
5. Enemy's Strategy:
6. Key Information: The early game is critical phase, and our limited unit count and resources indicate that we should prioritize both economy and army development. The ongoing Warpgate research suggests transition to more advanced unit production.
Suggestions:
1. Our Strategy: Continue with the Warpgate research as it will significantly improve unit production capabilities. Consider Chrono Boosting the Cybernetics Core to expedite this research.
2. Units and Buildings: Prioritize building more Zealots bolster your army. Also, expand your Pylon network increase your supply cap and ensure you don't 't get supply blocked.
28

---

3. Economy: Focus on worker production. Consider building more probes to saturate your mineral and gas collection, which will support both army and tech development.
4. Technology: After completing Warpgate research, evaluate your unit composition and decide which tech path to pursue based on the enemy's strategy. Consider building additional tech structures like Twilight Council Robotics Facility.
5. Keep eye the enemy's 's expansion and unit production. Prepare for potential Zer- gling attacks by having Zealots ready for defense.
Decisions:
0: <RESEARCH WARPGATERESEARCIS
<BUILD ZEALOT>
2: <BUILD PROBE>
3: <BUILD PYLON>
4: <CHRONOBOOST CYBERNETICSCORE>
% +9
Protoss cy cybem oor coore cumenily
bangy chrono bonsasii.
31
S18
154
A.I.1 (Hord)
36
225
318
3-2
20
Figure 17: Decision of LLM Agent: Chronoboost Cybernetics Core
29

---

F.3
Example 3
11 rosurs
At M42425 çame time, current StarCroft dituttion tollows:
04 25 game time our current StarCraft Situttion aol ows:
At 01:28 game time, current StarCreft S.0.0 fallowe:
Resources:
Resources:
Resourcass 04.25 worket supplyy
mineral 215
Jel:
mineral 235
supplyyuued suppry:
Guilding:
count
Bylon,
stargate
gas
pylon_ccunic
gateway coune
Usis
Unis
Units
gellwwy courE gwu.gtt
Zealx oount
bose pount 2
Beonnng.wwrreer.co 2ount:
Bylon, sooutt
gas buildingg....... counc:4 gatewayy count
In: procoss
hh-oracess:
planning gateway, count:
-parning robotica facility count:
planning wilgght coundil counc:
In- propreess
-parning stangati count: -paaninng wor 2
planning raboliss fusilty sount
p arninn gwteway, 1ount:
plamming statigate count
aanning,ti Dot wtw.counccccoooooooo Louncct
planning worier count:
Researth: -worpgata nww.arch statue
Research
Reseands
wwwrpaana sosorrchstatuucc
wwarggate httpattcccsorrh..ooo osearch_stonus
Cremyy iformation:
Enemyy Hnfomation:
Anttyyped 1
Enweny Informetione
HATCHERY.
Unttlypst EXTRACTO:: Unttyy http:/tpppppSSo///PPPPPOOOO
Un (Typel EX EXTRAOO
enemy, San.yUuntyyadssSAANUNRRO..
enomy htwpyy/innyyootsss///W......
Figure 18: L1 results of 04:25 period
LLM output:
1. Game Overview: At 04:25 game time, our situation is quite challenging. We have limited resources with 215 minerals and 128 gas. Our supply cap is at 54 with 16 supply left, and our army supply is just We currently have two bases.
2. Current Game Stage: Based on the limited resources and small army size, we are in the early game stage.
* 3. Our Situation:
3.1 Units and Buildings: We have modest army consisting of 4 Zealots. We are in the process of constructing additional Gateways, Robotics Facility, and Stargate.
3.2 Economy:
Our economy is struggling with minimal resources. We need to expand resource collection urgently.
3.3 Technology:
We have already researched Warp Gate technology, which is a posi- tive development. However, we lack diversity in our units and are still in the process of building advanced tech structures.
4. Our Strategy: It appears we are gearing up for tech-heavy approach with the construc- tion of advanced tech structures. However, our current economic state is not supporting this strategy well.
5. Enemy's s Strategy: The enemy seems to be focusing on expanding with Hatchery and Extractor but has also started building Spawning Pool, indicating the potential for Zerg unit production.
6. Key Information: The most crucial aspect at this moment is our resource scarcity. We need to address this issue immediately to support our tech-heavy strategy and build more substantial army to defend against potential enemy threats.
Suggestions:
1. Our Strategy: Given our low resources, it's crucial to prioritize resource expansion in the short term. Build additional Probes to maximize mineral and gas collection. Consider defensive strategy while we bolster our economy.
30

---

2. Units and Buildings:
Continue constructing the planned tech structures, but also priori- tize the construction additional Gateways for faster unit production. Warp-in Zealots as cost-effective defensive measure.
3. Economy:
Expand to new resource locations to increase income. Keep worker produc- tion consistent and utilize Chrono Boost Nexus to accelerate Probe production.
4. Technology:
Complete the construction of the Robotics Facility and Stargate unlock advanced unit options. Consider researching unit upgrades once these structures are opera- tional. Given the early game situation, these adjustments will help stabilize our economy and prepare for potential enemy threats.
Decisions:
0: <TRAIN PROBE>
<BUILD GATEWAY>
<EXPAND TO NEW RESOURCE LOCATION
3: <BUILD ROBOTICSFACILITY>
4: cCHRONOBOOST NEXUS>
The Proloss base Nexts is being
chrono boostrl
4:28
38
NEXUS
A.I.1 (Hord)
47 :04
235
352
24
Figure 19: Decision of LLM Agent: Chronoboost Nexus
31

---

E4
Example 4
rosurs
At 38 game time, our current, StarCroft
sitaition as follows:
At 1327 game time, our current StarCrott ait on followes
Resourcee:
Az 1528 game time, our currntt StarCrcft
Sib.t on is:25 followe:
Respurces:
-grmm lime: 13:27
Resourcase game. time: 13:29
minaral 1066
game tine: 1326
wor suppty 54
Building:
base souuri
onomy, un count
worker supply 53
mineral 1305
95p
Guildings:
Prylon counte
supoly kift:
supoly 255
enemyyyutts
suscly used: 140
-hhehld battery 4
my 57
gas, www.diiddinnn.c.
-yybor nitict
count
enemyyuuntts,, 10
galeway
buiteryy
-yybernetics, pore,
worp count
onomy JUN COUn
untty_untts,.om oount
Thilight www.occccoooooooooooooe Loount
base,
robotko, fwcility sount
Enemy inforatioon
cnomy un sount
Bylan,
pas, www.buiiiingg.oo count
Units:
enterry Unt.yyped
eneeryy UnilTyped OWERSERR:
galeeway pountt:
entrryy units SOUrE. 10
-nneryy Setpr.__nttyyettsooaarr0 CORSUPTOR:
Units
-olcossus & count
stalenr count
statgann, count:
ospanverr count. 2
roboties bay, 1ount
disuppto woun: 1
enemyy Unitlypetd Stw.://petSSPA/NPPPPPOOOO
enemy Unntlypt:/XXTAATTTTS
-nnmyy UD
-eneryy
-nnemy StT.TyaattsSPRRRRAWWWR...
-nnmyyy Sot.:y_UntyyattBBR000o0
Unitlypelt -onemyyUuttyypeeet
BROOCLORD:
stalker court
dpoupto sount 1
phoenk Dount: 2
enemyy units, count 10
colossur, count
observer pount:
vaidray. sount
2
voidray www.to 13
Figure 20: LI results of 13:28 period
LLM output:
1. Game Overview: At 13:28 game time, our situation is as follows: We have 54 workers, 1135 minerals, and 975 gas. Our supply cap at 155 with supply left, and our army supply is 87. We have six bases.
2. Current Game Stage: Based on the game time and resource availability, we are in the mid to late-game stage.
. 3. Our Situation:
3.1 Units and Buildings: We have a decent worker count and strong army compo- sition with Stalkers, Colossi, Disruptors, Observers, Phoenixes, and Void Rays. Our building count is also significant, with multiple pylons, gateways, robotics facilities, and robotics bay.
3.2 Economy:
Our economy is stable, with a good balance of minerals and gas. We have enough workers to maintain our resource collection efficiently.
We have already researched Warpgate technology, which essential for unit production. However, there is no information about further technological advancements or upgrades.
3.3 Technology:
4. Our Strategy: Our strategy seems be focused on strong ground and air army composi- tion, utilizing Stalkers, Colossi, Disruptors, Phoenixes, and Void Rays. We should continue to leverage our army strength and maintain a strong economy.
5. Enemy's Strategy: The enemy has a diverse unit composition, including Brood- lords, Overlord Transports, Corruptors, Overseers, and ground units. They seem be focusing on air superiority and potentially using Broodlords for high ground control.
The most crucial aspect at this moment is our strong army com- position, which includes anti-air units like Phoenixes and Void Rays. However, should be cautious of the enemy's Broodlords, which can pose significant threat to our ground forces. Additionally, we need to focus on further technological advance- ments and upgrades to stay ahead in the game. Suggestions:
6. Key Information:
32

---

1. Our Strategy: We should continue to strengthen our army composition by adding more anti-air units like Phoenixes and Void Rays. Additionally, we should consider countering the enemy's Broodlords with appropriate units or tactics.
2. Units and Buildings: Given the enemy's air dominance, it would be beneficial to increase the production of Phoenixes and Void Rays. Additionally, consider building more Stargates to support the production of these units. It may also be necessary to reinforce our ground forces with additional Stalkers and Colossi
3. Economy: Maintain a steady worker production and ensure efficient resource gath- ering. Consider expanding to new resource locations support the production of larger army.
4. Technology: Research upgrades that enhance the effectiveness of our units, such as air weapon and armor upgrades for Phoenixes and Void Rays. Additionally, consider researching extended thermal lance for increased Colossus range.
Decisions:
0: <TRAIN PHOENIX>
1: <TRAIN VOIDRAY>
2: <BUILD STARGATE>
3: <TRAIN STALKER>
4: <TRAIN COLOSSUS>
PRODUCTION
Protoss Countering Strategy: Producing Voidrays and Colossi against Zery Bruód Lords
C Xi
Zery Air A may: Brood Lords and Översoors
13:39
Protoss A irA my: Voidrays
154
265 11255
463
1231
61
93
6800
3200
861
Searics
Tich oato
Joan
R1a 106
(Hard)
113
326
60
4900
4250
Figure 21: Decision of L.MM Agent: Adapt strategy in response opponent's tacties.
33

---

G Questions and LLM Responses
In subsection we documented the responses from various LLMs, encompassing GPT3.5, GPT4, Claude2, and Bard, spanning five distinct question categories.
1. Q1:Basic Knowledge of StarCrafi2
(a) Are you familiar with StarCraft2?
(b) Could you provide overview of StarCraff,,ncclddnng its races, esports scene,and other pertinent details?
2. Q2:Racial Mechanics StarCraft2 Discuss the unique mechanisms of the following races:Zerg.Protoss and Terran.
3. Q3:Typical Strategies in StarCraft2 for Different matchups?
(a) TVZ:What's 's common strategy for Teeran vs Zerg?
(b) TVT:What's typical srategy in Terran Terran matchups?
(c) TVP:How do Terran players generally approach Protoss opponent?
(d) ZVZ:What's standard strategy Zerg vs Zerg battles?
(c) ZVP:What are Zerg strategies when facing Protoss?
(f) ZVT:How do Zerg Players counter Terran tactis?
(g) PVZ:How should Protoss approach Zerg opponent?
(h) PVP:What's the strategy for Protoss vs Protoss gams?
(i) PVT: do Protoss players generally strategize against Terran?
4. Q4:Ability to Outline Build Orders for Different Races. Can you provide standard build order for each race?
5. Q5: Classic Strategies and Counterplays in StarCraf12
(a) PVZ:What are effective strategies and counterplays against a roach-hydra attack?
(b) TVZ:How should Terran approach mutalisk-zergling banndiikk composition?Are there gambits or counter-strategen to consider?
(c) TVP: faced with an archon-zealot push, what are the bast strategies and counters for Terran?
G.1 Basic Knowledge of StarCraft II
Q1: Are you familiar with StarCraft II?
Yes, I'm familiar with StarCraft II. StarCraft a popular real-time strategy (RTS) video game developed and published by Blizzard Entertainment, It is the sequel to the original Star- Craft and was released in three separate episodes: Wings of Liberty, Heart of the Swarm, and Legacy of the Void. The game is set in a science fiction universe and features three distinct factions: the Ter- rans, the Zerg, and the Protoss, each with its own unique units and abilities. Players in StarCraft II engage in strategic and tactical battles, gathering resources, building armies, and competing against other players or Al opponents. The game has a strong competitive scene, with professional players and tournaments, and it has dedicated fanbase worldwide. If you have any specific questions about StarCraft II or need information on a particular aspect of the game, feel free to ask!
GPT3.5:
Yes, I'm familiar with "StarCraft II, the popular real-time strategy (RTS) game developed and published by Blizzard Entertainment. The game is sequel to the original "StarCraft" from 1998 and is in a distant part of the Milky Way galaxy the 26th century. It maintains the classic RTS elements of gathering resources, building bases, and commanding armies to defeat opponents. "StarCraft II" consists of three main installments representing each of the game's 's species: (1) "Wings of Liberty" (released in 2010) This is the base game where the primary campaign focuses on the Terrans, specifically on the character Jim Raynor, who leading a rebellion against the autocratic Terran Dominion. (2) "Heart of the Swarm" (released in 2013) This expansion focuses on the Zerg race and follows the story of Sarah Kerrigan (formerly a Terran who was transformed into the Queen of Blades, the leader of the Zerg) in her quest to reunite the Zerg S Swarm and exact revenge on the
GPT4:
34

---

Terran Dominion's 's emperor, Arcturus Mengsk. (3) "Legacy of the Void" (released in 2015) The final installment concentrates on the Protoss and features the character Artanis as he works to unite the Protoss and confront cosmic threat known as the "Dark Voice" or "Amon."
The game is recognized for its competitive multiplayer mode, where players from around the world compete in leagues, Each of the three races in the game (Terrans, Zerg, and Protoss) has its unique set of units and structures, which results in very different strategies and requires deep understanding of the game to master." "StarCraft II" has also been a significant part of the professional e-sports scene, with numerous tournaments worldwide, including the prestigious StarCraft II World Championship Series (WCS). The game's 's emphasis on quick decision-making strategic diversity, and real-time tactical execution contribute to standing as respected e-sport.
Claude2: Yes, I'm familiar with StarCraft 2! It's a popular real-time strategy video game devel- oped by Blizzard Entertainment. Here are some key facts about StarCraft 2:
It's the sequel to the original StarCraft game that came out in 1998. StarCraft was released in 2010.
There are three installments or chapters in StarCraft 2 Wings of Liberty, Heart of the Swarm, and Legacy of the Void.
Players can play as three factions or races the Terrans (humans), the insect-like Zerg, and the technologically advanced Protoss.
The gameplay involves gathering resources, building bases and armies, and battling opponents in real-time. The goal is to destroy the enemy base.
StarCraft 2 has a thriving eSports scene with many professional tournaments and leagues. Top players can earn significant prize money.
It's considered one of the greatest and most influential real-time strategy franchises date, known for its balanced asynchronous gameplay, strategic depth, and cinematic storytelling.
Blizzard continues support StarCraft 2 with updates, new co-op commanders, and balance changes to keep the multiplayer aspect fresh.
Let me know if you have any other specific questions about StarCraft 2! I'm happy to discuss strategies, lore, competitive play, or anything else related this classic RTS game.
Yes, I am familiar with StarCraft II. science fiction real-time strategy video game developed and published by Blizzard Entertainment. It was released worldwide in July 2010 for Microsoft Windows and Mac OS X. StarCraft II is the sequel to the 1998 game StarCraft, and is set four years after the events of the original game. StarCraft II features three playable races: the Terrans, the Zerg, and the Protoss. Each race has its own unique units and abilities, and the player must use their strengths to defeat their opponents. StarCraft II can be played in single-player mode, cooperative mode, or multiplayer mode. The single-player campaign in StarCraft Il is divided into three parts: Wings of Liberty, Heart of the Swarm, and Legacy of the Void. Each campaign follows the story of different character from one of the three races. The cooperative mode in StarCraft Il allows players to team up with other players complete missions against computer-conee computer-controlledd enemies. The multiplayer mode in StarCraft allows players to compete against each other in ranked unranked matches. StarCraft II is very popular game, and it has large and dedicated esports scene. StarCraft tournaments are held all over the world, and some of the best players in the world can millions of dollars in prize money.
Bard:
there anything else you would like to know about StarCraft II?
Q2:
Could you provide an overview of StarCraft II, including its races, esports scene, and other pertinent details?
Certainly! Here's overview of StarCraft II, including its races, esports scene, and other important details:
GPT3.5:
Game Overview:
35

---

Release: StarCraft II was released in three separate episodes. The first episode, "Wings of Liberty," was released in July 2010. The second episode, "Heart the Swarm," followed in March 2013, and the final episode, "Legacy of the Void," was released in November 2015.
Genre: StarCraft II a real-time strategy (RTS) game that focuses on resource manage- ment, base building, and strategic combat.
Developer and Publisher: Developed and published by Blizzard Entertainment.
The game is set in a futuristic science fiction universe and features three distinct races: Terrans, Zerg, and Protoss. Each race has its own unique units, buildings, and gameplay mechanics.
Setting:
Single Player: Each episode of StarCraft II has a single-player campaign that follows the story of one of the three races. The campaigns offer engaging storytelling and missions with varying objectives.
Races:
Terrans are humans who have been exiled from Earth and are known for their adaptability and use of advanced technology. They have wide range of mechanical units and structures.
Terrans:
Zerg:
Zerg are highly adaptable and insectoid alien race that can rapidly grow and over- whelm their opponents with numbers. They are known for their biological units and ability to spread creep.
Protoss: Protoss technologically advanced alien race with powerful psionic abilities. They have advanced units and structures, and their gameplay is focused on precision and strategy.
Esports Scene:
StarCraft II has thriving esports scene, with numerous tournaments and professional players from around the Wojod.Major tournaments include the World Championship Series (WCS), Intel Extreme Masters (IEM), GSL (Global StarCraft II League), and Professioon players and teams compete for significant prize money and recognition in these tournaments. The game's competitive nature and high skill ceiling make it popular choice for both casual and professional players.
Legacy and Impact:
StarCraft II is the successor to the original StarCraft, which is considered one of the most influential and iconic real-time strategy games of all StarCrttr II continued the legacy of its predecessor and expanded the game's ' lore and gameplay mechanics. The game has a dedicated fanbase and has had significant impact on the esports industry.
Community and Custom Content:
StarCraft II has vibrant modding and custom content community. Players can create and share custom maps, game modes, and mods through the StarCraft II Arcade,This community-drive aspect adds to the game's 's longevity and diversity of gameplay experiences. While my knowledge is based on information available up until September 2021, StarCraft has likely continued to evolve and develop in the years since. If you have specific questions need more recent information about the game or its esports scene, recommend checking official sources and community forums for the latest updates.
"StarCraft II" a monumental figure in the world of real-time strategy (RTS) games and e-sports. Developed by Blizzard Entertainment, it's known for its demanding skill level, strategic diversity, and an active professional scene. Here's 's a comprehensive overview:
GPT4:
1. Overview:
Released in 2010, 'StarCraft II" is set in science fiction universe in the 26th century and focuses on the interstellar struggles between three species: the Terrans (humans exiled from Earth), the Zerg super-species of assimilated life forms), and the Protoss (a technologically advanced species with psionic abilities).
2. Gameplay:
36

---

Players choose one of three races to control, each offering a unique array of units and technologies. Games typically involve gathering resources, expanding bases, and commanding armies defeat en- emies. The game's s asymmetrical balance is defining characteristic, with each race having different units, strengths, and weaknesses, requiring distinct strategies.
3. Races:
Terrans: Known for their adaptability and defensive capabilities, Terrans have versatile units and a unique building mechanic (buildings can lift off and move). They are human exiles from Earth. Zerg: The Zeng all about rapid proliferation and adaptation. They produce large numbers of units quickly and can evolve them into more advanced forms. They rely on overwhelming their opponents with sheer numbers and have the ability rapidly spread across the map. Protoss: Protoss units and buildings are generally more expensive (in terms of in-game resources) but highly powerful and efficient. They have strong shields, advanced technology, and can warp units directly onto the battlefield.
4. Campaigns:
"StarCraft II" was released as a trilogy, with each installment focusing on different race's perspec- tive in the ongoing interstellar war. "Wings of Liberty": Focuses on the Terran campaign. "Heart of the Swarm": Revolves around the Zerg campaign. "Legacy of the Void": Concludes with the Protoss campaign.
5. E-Sports:
Shortly after its release, "StarCraft II" exploded into the e-sports scene. It built on the legacy of the original "StarCraft" game, which was one of the first globally recognized e-sports. The game celebrated for its incredibly high skill ceiling and strategic depth, making professional matches fascinating for spectators. The "StarCraft II World Championship Series (WCS)" was the premier competitive circuit for professional "StarCraft II" players, featuring several regional events leading up to global year-end championship. South Korea, known for pioneering e-sports, has been partic- ularly prominent in the "StarCraft II" scene, producing numerous professional players who dominate in international competitions.
Community and Legacy:
"StarCraft II" has a strong community of players, fans, and dedicated content creators. The game's custom map editor has led variety of player-created content, adding to the game's longevity. In October 2020, Blizzard announced the end of active development of new content for "StarCraft II," shifting a maintenance mode for the game with focus balance and ecosystem health. Despite the reduction in developer-driven content, the community and e-sports scene remain active, with third perrtpprty organizers continuing to hold tournaments.
Free-to-Play Transition:
In 2017, "StarCraft II" became free-to-play, allowing players to access the original "Wings of Lib- erty" campaign and the multiplayer modes for free, which helped maintain the game's player base. The legacy of "StarCraft II" in the gaming community is marked by its technical and strategic depth, the skill demands from its players, and its significant impact on the broader e-sports industry.
Claude2: Absolutely, here's an overview of StarCraft II:
There are three playable races in StarCraft II the Terrans (humans), the insectoid Zerg. and the technologically advanced Protoss. Each race has distinct units, structures, abilities, and playstyles.
Races:
Terrans are versatile and mobile, with units like Marines, Tanks, and Battlecruisers. They can adapt quickly and have a lot of defensive and zone control options.
Zerg are focused overwhelm and expansion, capable of producing large armies rapidly. Key units are Zerglings, Roaches, Hydralisks and Mutalisks. They can overrun opponents through sheer numbers.
Protoss have advanced technology and psionic abilities. Their units like Stalkers, Immortals and Carriers slower but more powerful. They rely on coordination and positioning in engagements.
37

---

eSports Scene: StarCraft II has longstanding professional esports scene in Korea especially. Top players compete premier tournaments like GSL and Proleague for prize money and glory.
Popular pro players include Serral, Maru, Rogue, Dark and soO. Blizzard directly supports the scene through the ESL Pro Tour and other circuits.
The main competitive mode is Ivl, but 2v2, 3v3 and 4v4 team games are also popular. There is also a Co-op PVE mode where players team up against Al enemies. Custom games allow for kinds of player-created variants.
Game Modes:
Wings of Liberty, Heart of the Swarm and Legacy of the Void are the three installments that comprise StarCraft II. Each focuses on one race's campaign story while expanding multiplayer.
Expansions:
Let me know if you need any clarification or have additional questions!
Bard: Sure. Here is an overview of StarCraft II, including its races, esports scene, and other pertinent details:
Stces:StarCraft II features three playable races: the Terrans, the Zerg, and the Protoss. Each race has its own unique units and abilities, and the player must use their strengths to defeat their opponents.
Terrans: The Terrans are a versatile race that combines human ingenuity with advanced technology. They have a wide variety of units to choose from, including Marines, Siege Tanks, and Battlecruisers. Terrans are also known for their powerful mobile structures, such as Command Centers and Factories.
Zerg: The Zerg are wwarm-based race that overwhelms their enemies with sheer numbers. They have a variety of cheap and expendable units, such as Zerglings and Roaches, as well more powerful units such Hydralisks and Ultralisks. Zerg players are also known for their ability to quickly expand their bases and produce units.
The Protoss are highly advanced race that relies on powerful technology and psionic abilities. They have a limited number of units to choose from, but each unit is very powerful. Protoss players are known for their strong economies and their ability build powerful armies.
Protoss:
Esports scene
a large and dedicated esports scene. StarCraft tournaments are held all over the world, and some of the best players in the world can earn millions of dollars in prize money. biggest StarCraft II tournament of the year is the Global StarCraft II League (GSL). The GSL is premier tournament series that features the best players from all over the world. Other major StarCraft tournaments include the Intel Extreme Masters (IEM) and the World Championship Series (WCS).
StarCraft II
Other pertinent details
StarCraft II is a very challenging game, and it takes a lot of practice to master. The game is also very popular among competitive gamers, and it has a very active esports scrn are some other pertinent details about StarCraft II:
StarCraft II a free-to-play game. The base game includes the single-playe campaign for Wings of Liberty, as well as the multiplayer mode. The other two campaigns, Heart of the Swarm and Legacy of the Void, can be purchased separately.
StarCraft II has very active modding community. There are thousands of mods available for StarCraft II, which can add new units, maps, and gameplay mechanics to the game.
StarCraft is very popular game among streamers. There are many professional StarCraft II streamers who broadcast their gameplay to viewers on Twitch and YouTube.
If you are interested in learning more about StarCraft II, recommend visiting the official Blizzard website or watching some StarCraft II streams on Twitch or YouTube.
G.2 Racial Mechanics in StarCraft
Q: Discuss the unique mechanisms of the following races: Zerg, protoss and Terran
38

---

Certainly! Each of the three races in StarCraft II has its own unique mechanics and game- play style. Here's an overview of the racial mechanics for the Zerg, Protoss, and Terran factions: Zerg:
GPT3.5:
Creep Spread: Zerg players can spread "creep" across the map. Creep is slimy substance that not only provides vision but also enhances the movement speed of Zerg units when they are on it. Creep is generated by buildings called "Creep Tumors" and the primary Zerg building, the Hatchery or Lair.
Larva Mechanic: Zerg units are produced from Larvae, which spawned by Hatcheries and Lairs. The number of Larvae determines how quickly a Zerg player can build units. Managing Larvae effectively is crucial Zerg gameplay.
The Zerg rely heavily on biological units and are known for their ability to quickly swarm opponents with large numbers of units. Their units often have regenerative abilities and can be upgraded to various forms. Evolution Chambers: Zerg units can be upgraded Evolution Chambers, allowing players to adapt their units to counter their opponent's ' strategies.
Biological Units:
Protoss:
Warp-In Mechanic:
Protoss units can warp in directly onto the battlefield from Pylons Warp Gates, providing unique mobility advantage. However, this requires strategic placement Pylons and Warp Gates.
Shields: Many Protoss units have energy-based shields in addition to their regular health.
Shields regenerate over time, making hit-and-run tactics effective when used wisely.
Psionic Abilities: Protoss units often have psionic abilities that can used for offense or defense. Examples include Force Fields, which can block enemy units, and Psionic Storm, which deals area damage.
Warpgate Research: Protoss players can research the "Warpgate" technology. allowing them transform their Gateways into Warp Gates, significantly increasing their flexibil- ity in producing units.
Terran:
MULEs and SCVs: Terran players can call down MULEs (Mechanized Utility Drones) to quickly gather resources. SCVs (Space Construction Vehicles) can also repair buildings and mechanical units.
Terran buildings are often mobile, with the ability to lift off and land in new locations. This flexibility allows for repositioning and strategic play.
Building Mobility:
Supply Depots and Wall-offs: Terran players Supply Depots provide supply and can use them to create wall-offs to block enemy movement.
Terran Infantry: Terran infantry umits can be upgraded with a variety of weapons and equip- ment, making them versatile and adaptable different situations.
Siege Tanks: Terran players often employ Siege Tanks, which can transform into siege mode to deal massive area damage. This unit is cornerstone of Terran defensive strategies.
Each race in StarCraft II offers distinct and challenging gameplay experience, and mastering their unique mechanics is essential for success in the game. Players often choose a race that suits their pre- ferred playstyle, whether it's the swarm tactics of the Zerg, the advanced technology of the Protoss, or the adaptability of the Terran.
"StarCraft stands out in the RTS genre due to its distinct, asymmetric gameplay, mean- ing each race has unique mechanisms, units, and strategies that create deep, strategic environment. Here's ' detailed look at the specific racial mechanics for the Zerg, Protoss, and Terran factions:
GPT4:
1. Zerg:
39

---

Creep: One of the most distinctive aspects of the Zerg is their reliance Creep, living mat of bio-orgamic substance that spreads from their main structures. Zerg buildings, except for the Hatchery, must be built on Creep. It also allows Zerg units to move faster when they are on it, providing both strategic and defensive advantages.
Larvae and Macro Mechanics: Zergs uniquely produce units through Larvae generated by their primary structure, the Hatchery (which can be upgraded to a Lair and then a Hive for more advanced units). These Larvae morph into different units, meaning the Zerg can adapt their production in real-time as they see fit. The Queen unit can also spawn additional Larvae to speed up production, emphasizing the Zerg's swarm-oriented strategy.
Rapid Expansion: Zerg players often rapidly colonize areas of the map to gain more re- sources, using inexpensive units to overwhelm opponents. Their ability to quickly spread Creep helps in establishing new bases and fortifying territories.
2. Protoss:
Warp Gate Technology: definitive feature for the Protoss is their advanced technology, represented through the transformation of their Gateway buildings into Warp Gates. This allows the production and immediate deployment of units across the battlefield, provided there is power field (generated by structures like the Pylon). This ability is crucial for both reinforcing units during a battle and executing surprise attacks.
Shields and Health: Protoss units and buildings have layers of health: a regenerative shield and their actual health. This characteristic allows Protoss armies to be very resilient, shield points regenerate over time when not under attack.
Chrono Boost: The Protoss Nexus, the main base structure, has an ability called Chrono Boost that can be used to speed up the production of units, research, and buildings. This feature is integral to the race's adaptability, allowing them to respond quickly threats.
Terran:
Building Mobility: Terran structures, unique amongst the races, have the ability to lift off and relocate, allowing for strategic repositioning, surprise tactics, and base migration. This feature is particularly useful for avoiding threats, scouting, and creating forward unit production points.
MULEs and Macro Mechanics: Terran Command Centers can call down MULEs (Mining Unit, Lifting Equipment), temporary units that gather resources at much higher rate than SCVs, the standard worker unit. This ability provides significant boost the Terran economy over short periods, allowing for bursts of increased production.
Bunkers and Siege Warfare: The Terrans have a defensive style of play, with structures like Bunkers and units like the Siege Tank. Bunkers can be garrisoned with infantry units to improve their survivability and increase the Bunker's firepower. Siege Tanks have a transformation ability, allowing them to switch stationary siege mode with increased range and damage, ideal for both defense and area control.
Each race's unique mechanics require players adopt very different strategies for each, contributing to the game's s dynamic competitive environment. Mastering the utilization these racial featuresa- long with understanding those of potential opponentsis crucial for success in "StarCraft II."
Claude2:
Here's 's an overview of the unique racial mechanics for each of the three StarCraft II
races:
Zerg
Larva - Zerg units are produced from larva generated by Hatcheries. This allows the Zerg
player to quickly amass large armies.
Creep Zerg buildings must be built on creep, the purple biological carpet produced by Hatcheries and Creep Tumors. Units move faster on creep.
Inject Larva Queens can inject larva into Hatcheries to boost unit production. Proper injecting is essential for Zerg macro.
40

---

Many Zerg units can morph or evolve into more powerful variants. Examples: Banelings, Lurkers, Brood Lords.
Morphing
Protoss
Warp-in- Protoss can warp-in units directly onto the battlefield provided they have Pylon power source and Gateway/Wary Gate ready. Critical for reinforcements.
Chrono Boost The Nexus's Chrono Boost ability accelerates building and research times. Important for protoss economy and tech.
Psionics Many protoss units have psionic abilities like force fields, hallucination and psi storms that provide battlefield control.
Power Fields Pylons power protoss buildings and enable warp-in. Destroying Pylons cuts power and supply for an area.
Terran
MULES SCVs can build MULEs at Orbital Commands that harvest resources faster rate than SCVs. Boosts income.
Salvage Most Terran structures can salvaged to recoup some of the resources invested into them. Provides economic flexibility.
Add-ons- Buildings like Barracks and Factories can have Reactor or Tech Lab add-ons to enhance unit production capabilities.
Transforming Units Many Terran units like Hellions and Vikings can transform between modes adapt as needed during battle.
Let me know if you need any clarification or have additional questions!
Bard: Zerg
The Zerg are swarm-based race that overwhelms their enemies with sheer numbers. They have a variety of cheap and expendable units, such as Zerglings and Roaches, as well as more powerful units such as Hydralisks and Ultralisks. Zerg players are also known for their ability quickly expand their bases and produce units.
Here some of the unique racial mechanics of the Zerg:
Inject Larva: Zerg Hatcheries produce Larva, which can then be morphed into different types of units. The Inject Larva ability allows Zerg players to increase the production rate their Hatcheries.
Creep Spread: Zerg Creep is substance that spreads from Zerg structures and units. Zerg units gain movement speed and attack speed on Creep. Zerg players can use Creep to quickly move their armies around the map and to gain an advantage in battles.
Overlord Speed: Overlords are Zerg flying units that provide vision and can transport units. Zerg players can upgrade the speed of their Overlords, which allows them to quickly scout the map and to transport their armies to different locations.
Zerg Rush: Zerg Rush is a strategy where Zerg players produce a large number of cheap and expendable units early in the game and attack their opponent. Zerg Rushes can be very effective, but they can also be countered by players who are prepared for them.
Protoss
The Protoss are highly advanced race that relies on powerful technology and psionic abilities. They have a limited number of units to choose from, but each unit is very powerful. Protoss players are known for their strong economies and their ability to build powerful armies.
Here are some of the unique racial mechanics of the Protoss:
Warp Gate: Warp Gates are Protoss structures that allow Protoss players to warp in units directly from their Nexus. This allows Protoss players to quickly produce units any location on the map.
41

---

Chrono Boost is Protoss ability that can be used to speed the production of umits or the research of technologies. Protoss players can use Chrono Boost gain an advantage in the early game or to quickly produce a powerful army.
Chrono Boost:
Shields: Protoss units have shields, which are layer of energy that protects them from damage. Shields regenerate over time, but they can be depleted by enemy attacks. Protoss players can use their shields to their advantage by engaging in hit-and-run tactics or by using their shields to absorb damage while their units attack.
Psionic Storm: Psionic Storm Protoss spell that creates storm of psionic energy that damages enemy units in large area. Protoss players can use Psionic Storm to clear out enemy armies or damage enemy structures.
Terran
The Terrans are a versatile race that combines human ingenuity with advanced technology. They have wide variety of units choose from, including Marines, Siege Tanks, and Battlecruisers Ter- rans are also known for their powerful mobile structures, such as Command Centers and Factories.
Here are some of the unique racial mechanics of the Terrans:
MULE: MULES are Terran worker units that can be called down from Orbital Command Centers. M.U.L.E.s mine minerals at faster rate than SCVs, which can help Terran players to quickly build up their economies.
Supply Drops: Terran Supply Depots provide supply, which is needed to produce units. Ter- ran players can also call down Supply Drops from their Command Centers, which provides additional supply. This allows Terran players to quickly build up their armies.
Siege Tanks are Terran tank units that can transform into a siege mode. In siege mode, Siege Tanks have longer range and deal more damage. Terran players can use Siege Tanks to defend their bases or to attack enemy bases from a distance.
Siege Tanks:
Medivacs: Medivacs are Terran flying units that can transport units and heal them. Terran players can use Medivacs to quickly move their armies around the map and to keep their units alive in battle.
Each race in StarCraft II has its own unique strengths and weaknesses. Zerg players are known for their ability to quickly produce large armies, Protoss players are known for their strong economies and powerful units, and Terran players are known for their versatility and mobile structures. The best way to learn about the unique racial mechanics each race to experiment with them and to play against other players.
G.3 Typical Strategies in StarCraft II for Different Matchups
Q1: TvZ: What's a common strategy for Terran vs. Zerg?
GPT3.5: Terran vs. Zerg (TvZ):
Bio-Mech Composition: Terran often uses a combination of infantry (Marines, Marauders) and mechanical units (Siege Tanks, Hellbats) against Zerg. This composition allows for mobility and firepower.
Harassment: Terran players often employ drop tactics, using Medivacs to drop units into Zerg bases to disrupt their economy and force them split their forces.
Widow Mines: Widow Mines can be used to deter Zergling and Baneling attacks, and they can also be dropped into Zerg mineral lines to deal economic damage.
A common strategy involves harassment to prevent the Zerg from comfortably expanding and massing their forces. Terrans might use Hellions to attack Zerg's 's worker units move units behind enemy lines. The overall aim to control the pace, forcing Zerg to react to Terran threats while building a strong economy and army for a decisive push.
GPT4:
or dropships to
42

---

Claude2: Terran often opens with Reaper expand into Hellions for map control and harassment. This transitions to Marine/Medivae for mobility and bio strength. Tanks added for splash against Zerglings.
Bard: A common strategy for Terran Zerg is to go for macro build, such Bio (Marines, Medivacs, and other support units) or Mech (Siege Tanks, Hellions, and other mechanical units). Terran players can also try to rush their Zerg opponent with early aggression, such Reaper Rush or Marine/Marauder Rush.
Q2: TvT: What's typical strategy in Terran vs. Terran matchups?
GPT3.5:
Tank Warfare: TvT often involves tank battles. Players set up Siege Tanks defensively and try to gain positional advantages while avoiding being caught unsieged.
Viking Control: Air control is crucial in TvT. Vikings are used to gain air superiority and
vision for Siege Tanks. Raven usage for interference matrix also be key factor.
Bio vs. Mech: Terran players can choose between a bio-heavy composition (Marines, Marauders) and mech-heavy composition (Siege Tanks, Hellions). The choice can depend on personal preference and map.
TvT often revolves around positioning and tank warfare. Siege Tanks are critical for area control, and players jockey for position and map control. Air superiority via Vikings is also crucial provide vision and control. Since all units available both sides, scouting, deception, and adaptability are key.
GPT4:
Both players open Reaper expand for scouting information. Focus is securing ad- ditional bases for economy. Tanks, Vikings and air control are critical for positioning and engage- ments.
Claude2:
Bard: A typical strategy in Terran vs. Terran matchups go for macro build, such as Bio or Mech. Terran players can also try to scout their opponent's 's build early and counter it with a specific composition. For example, Terran player sees that their opponent is going for Bio, they can counter with Siege Tanks and Hellions.
Q3: TvP: How do Terran players generally approach Protoss opponent?
GPT3.5:
Bio Play: Terran often employs bio compositions (Marines, Marauders, Medivacs) against
Protoss. Micro-intensive battles and mobility are key.
Ghost Usage: Ghosts can snipe haluevalue Protoss units like Templars and Observers. They also have EMP drain Protoss shields and energy.
Liberator Zones: Liberators can establish zones of control and disrupt Protoss positioning with their anti-ground mode.
Terrans often use a mixture of bio units (Marines and Marauders) supplemented with Medi- vacs for healing and mobility against Protoss. Harassment with drops disrupt the Protoss economy and tech is common. Terrans need to be aggressive, as a fully teched and maxed Protoss army can be difficult defeat head-on.
GPT4:
Claude2: Terran applies early pressure with units like Reapers, Hellions and Widow Mines. Goal to slow Protoss economy and tech while expanding themselves. Marines and Ghosts counter late-game Protoss power units.
Bard: Terran players generally approach a Protoss opponent by trying to expand their economy and build up a strong army. Terran players can also try to rush their Protoss opponent with early aggression, such as Reaper Rush or Hellion Rush.
43

---

Q4: ZvZ: What's a standard strategy in Zerg vs. Zerg battles?
GPT3.5:
Brggling-Baneling Wars: ZvZ is known for Zerglings and Banelings clashing. Players aim to control these units efficiently and gain map control.
Roach Timing Attacks: Roach-based strategies can be effective for pushing across the map and pressuring the opponent.
Mutalisk Play: Mutalisks are used for harassment and map control in the mid-to-late game.
ZvZ can be very volatile and often characterized by early aggressive plays, like Zergling and Baneling skirmishes. Map control via scouting with Overlords is crucial to see incoming threats. Due to the mirror nature, small advantages can escalate quickly, and tech switches counter the opponent's unit composition are common.
GPT4:
Claude2: Quickly droning up to bases while producing defensive units like Queens, Tech to either Roaches for safety or Mutalisks for harassment. Late-game transition to Brood Lords
Bard: A standard strategy in Zerg vs. Zerg battles is to go for a macro build, such as Mu- talisk/Zergling or Roach/Hydra. Zerg players also try rush their opponent with early ag- gression, such as a Zergling Rush or a Baneling Rush.
Q5: ZvP: What are Zerg strategies when facing Protoss?
GPT3.5:
Hydralisk and Lurker Play: Hydralisks and Lurkers are often used against Protoss, as they can deal with Gateway units effectively.
Creep Spread: Zerg players work to spread creep to control the map and gain vision, which is crucial in ZvP.
Tech Switches: Zerg may transition between unit compositions, such as Roach-Ravager into Brood Lords or Ultralisks, to keep Protoss players guessing.
Zerg players aim out-expand their Protoss rivals early on, using units like Zerglings and Roaches to defend against early aggression. They must be wary of Protoss area-of-effect (AoE) damage from units like High Templars and Colossi, often opting for tech like Corruptors or Vipers to neutralize key threats, and tech switches to keep Protoss guessing.
GPT4:
Defend early Protoss pressures like Adepts while droning up. Roach/Ravager with In- festors to counter Immortals, or Hydra/Lurker for anti-air. Vipers critical for late-game.
Claude2:
Bard: Zerg strategies when facing Protoss typically involve trying to overwhelm their opponent with large army of cheap and expendable units, such as Zerglings and Banelings. Zerg players can also try to flank their Protoss opponent with Mutalisks or to use their Hydralisks to pick off Protoss units from distance.
Q6: ZvT: How do Zerg players counter Terran tactics?
GPT3.5:
Ganeling-Mutalicn Zerg players often use Banelings and Mutalisks for harassment and map control. Banelings can be used to deal with Terran infantry.
Infestors: Infestors can be used to fungal growth Terran bio units and neural parasite pow- erful Terran units like Siege Tanks or Thors.
Brood Lords and Ultralisks: Late Late-game 7vT often involves transitioning into Brood Lords and Ultralisks for power and durability.
44

---

GPT4: Zerg often faces bio-ball compositions (Marines, Marauders, Medivacs) or mechanical units (like Tanks and Thors). Zerg needs to expand quickly, defend against harassment, and may use Banelings, Zerglings, and Hydralisks for defense transitioning to more robust units like Ultralisks or Brood Lords depending on Terran composition.
Ling/Bane for Bio defense. Fast spire into Mutalisk harass. Transition to Ultralisks supported by Queens and Infestors for the often mech-heavy Terran end-game.
Claude2:
Bard: Zerg players counter Terran tactics by using their superior mobility and economy their advantage. Zerg players can also use their powerful units, such as Ultralisks and Brood Lords, to defeat Terran armies
Q7: PvZ: How should Protoss approach a Zerg opponent?
GPT3.5:
Gateway Expand: Protoss players often open with Gateway expansion, using units like Adepts and Stalkers to control the early game.
Tech Choices: Protoss can opt for Stargate play (Phoenixes or Oracles) for harassment, or they may go for a Robo facility for Immortals or Disruptors.
High Templars: Templars are used for their Psionic Storm ability deal with large Zerg armies.
Protoss players must secure solid economic base while defending against Zerg's potential for rushes and harassment. Walling off chokepoints with buildings and using units like Zealots and Sentries to defend common early game. They rely on powerful AoE units (like Colossi and High Templars) later in the game, often leveraging tech advantages for strong pushes.
GPT4:
Claude2: Fast expand into Immorta//Archon/Chargll to defend Zerg attacks. High Templar for storms against Hydras. Skytoss deathball critical for late game.
Bard: Protoss players approach Zerg opponent by trying to expand their economy and build up a strong army powerful units, such as Zealots, Stalkers, and Immortals. Protoss players can also use their Colossi to deal damage to Zerg units from a distance.
Q8: PyP: What's the strategy for Protoss vs. Protoss games?
GPT3.5:
Stalker-Sentry: Early PvP often features Stalkers and Sentries for map control and defense.
Disruptors are key unit in PvP, used for their ability to deal massive area damage to enemy units.
Disruptor Play:
Blink Stalker Micro:
Micro-mannn merro-management of Blink Stalkers can be a deciding factor in PvP
engagements.
Early game revolves around preventing enemy harassment and scouting to anticipate tech choices. The mid-game often sees lot of positioning battles and potential for blink Stalker ha- rassment. Since Protoss units are powerful but costly, efficient trades and protecting key units are paramount, with late-ge ttte-game often seeing advanced units like Carriers Tempests.
GPT4:
Claude2: Robo for Immortals or fast expand with Blink Stalkers. Disruptors and High Templar for splash against massed Gateway units. Whoever gets Tempests wins the late game.
Bard: The strategy for Protoss vs. Protoss games typically involves trying to scout their oppo- nent's build early and counter it with a specific composition. For example, a Protoss player sees that their opponent is going for Colossi build, they counter with Immortals and Void Rays.
45

---

Q9: PvT: How do Protoss players generally strategize against Terran?
GPT3.5:
Adept Openings: Protoss may open with Adepts for early map control and harassment.
Colossi and/or Storm: Colossi are strong against Terran bio, while High Templars with Storm can deal with clumped bio forces.
Disruptors: Disruptors can be used to disrupt Terran positioning and force engagements.
Protoss strategies often involve careful defense against early Terran aggression, using units like Stalkers and Sentries, while advancing tech. Harassment with units like Oracles or Phoenixes can keep Terran unbalanced. the later game, Protoss looks to leverage high-tech units (like Colossi, High Templars, or Carriers) break through Terran's defenses.
GPT4:
Claude2: openings to secure expand. Robo for Observer scouting and Immortals against Terran vehicles. Storms and Archons counter Marine/Marauder bio play.
Bard: Protoss players generally strategize against Terran by trying to expand their economy and build up strong army of powerful units, such Zealots, Stalkers, and Immortals. Protoss players can also use their Colossi to deal damage to Terran units from a distance.
G.4 Ability to Outline Build Orders for Different Races
Q:Can you provide standard build order for each race?
GPT3.5:
Terran Standard Build Order:
Opening: Start with an SCV and send it to gather minerals.
Supply Depot: Build Supply Depot at 14 supply to ensure you don't get supply blocked.
Barracks: Build a Barracks at 16 supply.
Refinery: Take Refinery at 17 supply gas income.
Orbital Command: When the Barracks complete, upgrade your Command Center to an Orbital Command.
Reaper: Train Reaper from the Barracks for scouting and early map control.
Factory: At 100 gas, build a Factory.
Second Refinery: Take a second Refinery after the Factory.
Starport: Build Starport when the Factory is complete.
Tech Lab: Attach a Tech Lab to the Barracks.
Marine Production: Start producing Marines from the Barracks.
Reactor: Attach a Reactor to the Factory.
Medivac: Build Medivac from the Starport.
Tech Choices: Depending your strategy, you can go for upgrades, more production facilities, or expand.
Protoss Standard Build Order:
Opening: Start with a Probe and send it to gather minerals.
Pylon: Build a Pylon at 14 supply to avoid getting supply blocked.
Gateway: Create a Gateway at 16 supply.
Assimilator: Take an Assimilator 16 supply for gas income.
Cybernetics Core: Build Cyberneties Core when the Gateway is complete.
Pylon: Construct another Pylon at around 20 supply.
46

---

Adept or Stalker: Train an Adept or Stalker from the Gateway for early defense map control.
Nexus: Expand to natural expansion at around 21 supply.
Warp Gate Research: Research Warp Gate technology the Cyberneties Core.
Additional Gateways: Add more Gateways and Robotics Facility or Stargate based on your strategy.
Tech Choices:
Depending on your strategy, go for units like Stalkers, Adepts, or Oracles, and consider building more production structures.
Upgrades: Start researching upgrades at the Forge.
Expand: Consider taking a third base.
Zerg Standard Build Order:
Opening: Start with a Drone and send it to gather minerals.
Overlord: Build an Overlord at 13 supply.
Drones: Continue to produce Drones.
Hatchery: At supply, send a Drone to build a Hatchery at your natural expansion.
Extractor: Take an Extractor
supply.
Spawning Pool: Build a Spawning Pool 17 supply.
Overlord: Create another Overlord around 19 supply.
Queen: When the Spawning Pool is complete, produce Queen from your main Hatchery.
Zerglings: Train set of Zerglings for scouting and defense.
Extractor: Take second Extractor at your natural.
Evolution Chamber: Build an Evolution Chamber to start upgrades.
Additional Queens: Create more Queens for larva injects and creep spread.
Tech Choices: Depending on your strategy and opponent, go for Roaches, Banelings, or additional bases.
Lair: Start a Lair tech structure when you have the resources.
Further Expansion: Consider expanding to third base.
These build orders provide foundation for each race, but remember that flexibility and adaptation are key in StarCraft II. Adjust your build order based on scouting information and your opponent's actions maximize your chances of success.
Certainly, "StarCraft II" revolves heavily around build orders, which are pre-determined sets actions aimed at maximizing efficiency, economy, and production. These build orders set the stage for your mid-game and late-game strategies. Below are standard build orders for each race, which are foundational and can be adapted based on opponents' actions. Remember, the numbers represent the total worker count you should have when you begin each step.
GPT4:
Terran (Standard Opening):
14/15: Supply Depot
16/23: Barracks
16/23: Refinery
19/23: Orbital Command (immediately after Barracks completion)
20/23: Reaper (for scouting and slight harassment)
21/23: Command Center (on the low ground if you't 're feeling safe)
21/23: Factory (after Reaper starts)
23/31: Supply Depot
47

---

up:
Depends on scouting information (could include more Barracks, a Starport, or additional expansions and tech buildings).
Zerg (Hatch First Opening):
13/14: Overlord
17/22: Hatchery (expand)
18/22: Extractor (gas)
17/22: Spawning Pool
19/22: Overlord
20/22: Metabolic Boost (Zergling speed, once have 100 gas)
21/30: Queen x2 (once Spawning Pool is done)
30/36: Zerglings or other defensive units depending on what you scout
Follow-up: Based on the opponent's build, you might add additional bases, tech up (c.g.. Roach Warren, Lair), build more combat units.
Protoss (Standard Gateway Expand):
14/15: Pylon
16/23: Gateway
17/23: Assimilator
20/23: Nexus (expand)
20/23: Cybernetics Core
21/31: Assimilator
22/31: Pylon
23/31: Adept or Stalker (after Cybernetics Core finishes for defennse/scoutnng
Follow-up: Varied based scouting but might include additional Gateways, Robotics Facility, or Stargate, and tech advancements like Warp Gate research.
These build orders basic openings designed for beginners intermediates and meant to establish your economy while keeping you safe in the early game. They are not rigid play patterns. "StarCraft II" is dynamic game, and strategies need to adapt opponent actions. Effective scouting and understanding of your opponent's strategy are crucial, and as you gain more experience, you'll begin learn how alter your build order in response to what you't re seeing on the battlefield.
Claude2: Here are some standard opening build orders for each race in StarCraft II:
Terran Build Order:
14 Supply Depot
16 Barracks
Refinery
@ 100% Barracks Reaper and Orbital Command
22 2nd Command Center
@ 400 minerals
2nd Barracks
This gets an early Reaper for scouting and harassment, while using the Orbital Command for extra economy and flexibility. The 2nd CC and Barracks enable solid 2-base infrastructure.
Zerg Build Order:
17 Spawning Pool
17 Extractor
48

---

17 Hatchery
100% Spawning Pool 2 Queens
3rd Hatchery
@ 100 gas Metabolic Boost (Zengling speed)
@ 100% 3rd Hatch 2 more Queens
This provides quick Queen defense, fast Zergling speed, and lots of Larva for droning with the fast 3 Hatcheries.
Protoss Build Order:
14 Pylon
Gateway
17 Assimilator
20 Nexus
20 Cybernetics Core
22 Pylon
@ 100% Cybernetics Core Adept (or Stalker)
This gets expand into early tech for defense and scouting. The Adept provides psionic pressure and intel gathering. The build has potential to transition into several tech paths.
Let me know you need any clarification or have additional questions! Providing build orders is great way to demonstrate StarCraft knowledge.
Bard: Sure, here a standard build order for each race in StarCraft II: Terran
Bio Build Order:
Supply Depot
18 Barracks
19 Refinery
20 Supply Depot
Barracks
22 Refinery
23 Factory
24 Supply Depot
Command Center (Second base)
26 Refinery
27 Engineering Bay
28 Supply Depot
29 Refinery
30 Supply Depot
31 Starport
32 Supply Depot
Mech Build Order:
17 Supply Depot
Refinery
19 Supply Depot
20 Factory
49

---

21 Supply Depot
22 Refinery
23 Barracks
24 Refinery
25 Command Center (Second base)
26 Refinery
27 Supply Depot
28 Engineering Bay
29 Supply Depot
30 Refinery
31 Supply Depot
32 Starport
Zerg
Mutalisk/Zergling Build Order:
17 Hatchery
Gas
Spawning Pool
20 Extractor
21 Overlord
22 Extractor
23 Overlord
24 Lair
25 Overlord
26 Overlord
27 Hydralisk Den
28 Overlord
29 Overlord
30 Overlord
Spire
Roach/Hydra Build Order:
17 Hatchery
Gas
Spawning Pool
20 Extractor
21 Overlord
22 Extractor
23 Overlord
24 Lair
25 Overlord
26 Overlord
27 Hydralisk Den
28 Overlord
50

---

29 Evolution Chamber
30 Overlord
31 Overlord
Protoss
Zealot/Stalker Build Order:
Gateway
18 Probe
19 Pylon
20 Forge
21 Gateway
22 Cybernetics Core
23 Pylon
24 Gateway
25 Gateway
26 Gateway
27 Stargate
28 Pylon
29 Pylon
Colossus Build Order:
17 Gateway
18 Probe
Pylon
20 Forge
Gateway
22 Cybernetics Core
23 Pylon
24 Gateway
25 Gateway
26 Gateway
Robotics Bay
28 Pylon
29 Pylon
These are just standard build orders, and you can adjust them to fit your own playstyle. It is also important to scout your opponent's build early and counter it with a specific composition.
G.5 Classic Strategies and Counterplays in StarCraft II
Q1: PvZ: What are effective strategies and counterplays against roach-hydra attack?
51

---

GPT3.5: PvZ: Against Hayra Attack:
Strategy for Protoss:
Force Fields: Use Sentries to create Force Fields to segment the Hyych army, pre- venting from engaging as a whole.
Colossus units deal area damage and are effective against clumps of Roaches and Hydralisks.
Colossi:
Immortals: Immortals are strong against Roaches due to their bonus damage against ar- mored units.
Archons: Archons have splash damage and can help clear out Hydralisks.
Counterplay for Zerg:
Tech Hive: Transition Hive tech for units like Vipers, which can abduct key enemy units or use Blinding Cloud to reduce the effectiveness of enemy ranged units.
Hydra Speed Upgrade: Research Muscular Augments for Hydralisks to improve their mo- bility.
Burrow: Use Burrow for Roaches save them from being picked off and to regain health.
Drop Play: Drop Roaches Hydralisks into the Protoss main base for harassment to divert their attention. Creep Spread: Maintain good Creep spread to ensure mobility and vision.
GPT4: When Protoss players face Zerg opponent focusing Roach-Hydra, they're 're dealing with a versatile force that can be potent in a head-on fight due to its range, damage, and health, particularly in the mid-game.
Strategic Positioning: Force engagements in narrow passages where the Zerg cannot lever- age their numbers. Simultaneously, avoid open areas where the Zerg army can fully sur- round your forces.
These units are crucial. Their Psionic Storm ability deals massive dam- age to biological units, which makes them particularly effective against Hydralisks and clumped-up Roaches.
High Templars:
With their extended thermal lance upgrade, they can exploit their superior range and area-of-effect damage to decimate tightly packed Roach-Hydra armies.
Colossi:
Sentries: Force Fields can cut off portions of the Zerg army, preventing them from engaging all at once, or Guardian Shield can reduce incoming ranged damage.
Harassment: Divide the Zerg's 's attention. Warp Prisms, Zealot run-bys, and Dark Templars can force the Zerg player to split their army, making it less effective in a direct engagement.
Against Roach-Hydra, go Immortal/Archon with chargelots. Immortals counter Roaches, Archons counter Hydras. Focus fire with Immortals,an aamoortallllterraivvelyyy fast tech to Colossus. Use Sentries for forcefields to split Zerg army. Expand behind and get High Templar for storms.
Claude2:
Bard: Effective Strategies:
Go for Colossus build. Colossi very effective against Roaches and Hydras, and they can also deal damage to Zerg structures.
Use Void Rays to pick off Roaches and Hydras from a distance. Void Rays are also very effective against Zerg Overlords, which can help you to deny your opponent vision and to make it more difficult for them to attack you.
Use Warp Gates to quickly produce units and to respond to Zerg attacks. Warp Gates allow you to warp in units directly from your Nexus, which can help you to quickly build up your army and to defend your base. Effective Counterplays:
Use Observers to scout your opponent's 's base and to see what kind of army they are building. This will give you time prepare for their attack and to build the appropriate counter-units.
52

---

Ultralisks are powerful melee units that can be effective against Use Pylons to power your units and to build Shielding Batteries. This will help to protect your units from Zerg attacks.
Use Foree Fields to block off Zerg units and to create chokepoints. This will make it more difficult for your opponent to attack your base.
Q2: TvZ: How should Terran approach muttaiis--zergiin--annlnnn composition? Are there gambits or counter-strategies to consider?
GPT3.5: TvZ: Against Baallisk-Zerging-BBaa Composition:
Strategy for Terran:
Thors: Thors are strong anti-air units that can target Mutalisks effectively,
Widow Mines: Widow Mines can deal significant damage clumps of Mutalisks if prop- erly positioned.
Marine Splittig: Split your Marines to minimize the impact of Baneling detonations.
Missile Turrets: Build Missile Turrets in key locations to protect against Mutalisk harass- ment.
Vikings and Liberators: Vikings can help control the air, and Liberators can provide anti- ground support.
Counterplay for Zerg:
Baneling Drops: Use Overlords to drop Banelings onto clumps of Marines.
Transition to Ultralisks:
Terran infantry.
Corruptors: Corruptors can be used to target Thors and Vikings and reduce the anti-air capabilities of the Terran.
Infestors: Fungal Growth can be used to lock down Marines and other units.
Creep Spread: Maintain Creep spread to gain map control and mobility.
The Leaa-Linn-BBnee composition a highly mobile harassment force that seeks to control the map, prevent expansions, and pick off stray units. Dealing with it requires a mix of solid defense and map awareness.
GPT4:
Mine: Marines, with their high DPS, effective against Zer- glings and Mutalisks, especially when combined with Medivacs for healing. Widow Mines add layer of defense against Banelings and can devastate clumped Mutalisks.
Thors and Liberators: Thors help deter Mutalisks with their strong anti-air attack, pecially High Impact Payload mode. Liberators can zone out Mutalisks and provide support against ground units.
Base Defense: Well-placed Missile Turrets and Bunkers help defend against harass- ment. Building walls at choke points prevent Zergling run-bys.
Map Control: Establish map control with Marine or Hellion squads and maintain good vision to enemy movements. This approach helps in preparing for incoming attacks and catching the Zerg army out position.
Marines, Thors and Widow Mines counter Mutalisks. Thors splash damage and Widow Mine hits punish clumped Mutalisks. Turrets at mineral lines prevent economic harassment, Bio- mine with Medivacs can pick apart Zerglings. Siege Tanks zone Banelings.
Claude2:
Bard:
Effective Strategies:
Go for Bio build. Bio units are very effective against Mutalisks and Zerglings, and they can also deal damage Zerg structures.
53

---

Use Siege Tanks to deal damage to Mutalisks and Zerglings from distance. Siege Tanks are also very effective against Zerg Overlords, which can help you deny your opponent vision and to make it more difficult for them to attack you.
Use Medivacs to heal your units and to transport them around the map. Medivaes can also be used to drop your units into the middle of a Zerg army, which can be a very effective way to engage your opponent. Effective Counterplays:
Use Missile Turrets and Bunkers to defend your base from Mutalisk attacks. Missile Turrets and Bunkers are both very effective against Mutalisks, and they can also be used defend your workers and your structures.
Use Ravens to detect Mutalisks and to cast Interference Matrix on them. This will make it more difficult for your opponent control their Mutalisks and attack you.
Use Vikings to shoot down Mutalisks. Vikings the only Terran unit that attack air units without moving, which makes them very effective against Mutalisks.
Q3: TvP: When faced with an archon-zealof push, what are the best strategies and counters for Terran?
GPT3.5: TvP: Against Mechon-Zealos Push:
Strategy for Terra::
Kite and Focus Fire: Kite the Zealots with your ranged units (e.g., Marines and Marauders) and focus fire on Archons.
Medivac Micro: Use Medivacs to pick up and reposition units to avoid the melee engage- ments.
Widow Mines: Position Widow Mines hit the Zealots and soften the Archons.
Ghosts: EMP from Ghosts can drain the shields of Protoss units, including Archons.
Counterplay for Protoss:
Force Fields: Create Force Fields to segment the Terran army and prevent kiting.
Immortals: Immortals are strong against Archons due their bonus damage against ar- mored units.
High Templars: Use Psi Storm to deal AoE damage to the Terran infantry.
Air Units: Consider incorporating air units like Void Rays or Phoenixes for added versatil- ity.
Guardian Shield: Use the Guardian Shield ability from Sentries to reduce incoming ranged damage.
Remember that these strategies and counterplays Craft II. The specific outcome of battles can depend on micro-management, unit composition, up- grades, and positioning, so always be prepared adjust your tactics based on the evolving situation in the game.
exhaustive, and adaptability is key in Star-
GPT4: Archon rechon-Zealot (often with some Immortals) powerful, close-quarters Protoss army composition that relies on brute force and strong front-line units.
Ghost Usage: Ghosts" EMP Round is highly effective draining the shields of Archons, Zealots, and Immortals, significantly weakening their upfront staying power in battles.
Liberators and Siege Tanks: The use of Liberators with Defender Mode and Siege Tanks can create a zone of control that Zealots and Archons cannot approach easily. Kiting into these zones can decimate a Protoss anmy that's chasing.
Micro-intensive Kiting: Using bio forces (Marines and Marauders) supplemented with Medivacs for healing, a player can continuously kite against Zealots and Archons, which lack ranged attacks.
54

---

Sim City: Terran can build in way that chokes off paths (using Supply Depots, Bunkers, etc.), restricting the mobility and attack angles of melee units like Zealots and forcing Archons to funnel through tight spaces where they can't leverage their splash damage.
Each of these scenarios emphasizes not just the composition of your forces but also the importance of control, positioning, adaptability, and game sense. "StarCraft II" is game of constant decision- making, and successful players must be able to read their opponents and adjust their strategies on the fly.
Claude2: Marines and Marauders counter Zealots, while Ghosts and Liberators counter Archons. Kite with bio and emp Archons. Siege Tanks in tank mode punish Zealot chargelots. Well-spread mines damage clumped Zealots. Scan aggressively to preemptively snipe Observers and enable Widow Mines.
Bard: Effective Strategies:
Go for Bio build. Bio units very effective against Zealots and Archons, and they also deal damage to Protoss structures.
Use Siege Tanks to deal damage to Zealots and Archons from a distance. Siege Tanks are also very effective against Protoss Colossus, which can help you to defend your base and to make it more difficult for your opponent to attack you.
Use Medivacs heal your units and to transport them around the map. Medivacs can also be used to drop your units into the middle of a Protoss army, which can be very effective way to engage your opponent.
Effective Counterplays:
Use Missile Turrets and Bunkers to defend your base from Archon and Zealot attacks. Missile Turrets and Bunkers both very effective against Zealots and Archons, and they can also be used to defend your workers and your structures.
Use Scans to detect Protoss units and to see what kind of army they are building. This will give you time to prepare for their attack and to build the appropriate counter-units.
Use Snipers to pick off Protoss units from a distance. Snipers are very effective against Zealots and Archons, and they can also be used to deal damage Protoss Colossus.
55

---

H
NeurIPS Paper Checklist
H.1
Claims
Answer: [Yes]
Justification: The abstract and introduction clearly outline the main claims and contributions of developing new method for integrating LLMs into RTS gaming, partieularly StarCraft II, and improving model interpretabiliity These sections accurately reflect the scope of the research and its applicability other complex recision-makingg environments
H.2 Limitations
Answer: [Yes]
Justification:
TextStarCraft II's limitations include its reliance on non-visual data inputs, the use of scripted miero policies, and constrained performance due to resource limitations. For more detailed information, please refer to the Discussion section g
H.3 Theory Assumptions and Proofs
Answer: [NA]
Justification:
The paper primarily focuses on experimental results and method development rather than theoretical proofs.
H.4
Experimental Result Reproducibility
Answer: [Yes]
The paper details all necessary configurations, methodologies, and the TextStarCraft environment used, ensuring reproducibility. Additionally, we have open open-sourced the code, data, and demo, providing full access for researchers to replicate and verify our results comprehensively.
Justification:
H.S
Open access to data and code
Answer: [Yes]
Justification:We have provided open access to our complete source code and dataset, ensur- ing full reproducibility of our experiments. All resources, including training and testing scripts, are available in a publicly accessible repository linked in the supplemental materials of our paper. We also provide the anonymous l www.oouss.oppoooooooooooo science/r/ md.
H.6
Experimental Setting/Details
Answer: [Yes]
Justification: Our experimental setup thoroughly detailed in Section A. including comprehensive descriptions of agent and opponent selection, map choice, parameter settings, and more, ensuring robust and reproducible experimental environment.
H.7 Experiment Statistical Significance
Answer: [Yes]
Justification:
Our paper presents error bars calculated as the standard deviation across multiple experiment runs, capturing variability due initialization. These error bars are clearly displayed and annotated in the experimental result tables and figures.
H.8
Experiments Compute Resources
Answer: [Yes]
56

---

Justification: we have introduced our experiments compute resources, detailed in A
H.9
Code Of Ethics
Answer: [Yes]
Justification: Our research complies with the NeurIPS Code of Ethics, ensuring ethical standards and transparency in all experimental procedures, with measures in place to mitigate any potential risks, detailed M
H.10 Broader Impacts
Answer: [Yes]
Justification::Th paper discusses both positive and negative societal impacts of using LLMs in strate- gic environments, detailed in
H.11 Safeguards
Answer: [NA]
Justification:
Our research does not introduce data or models with high risk for misuse. The project focuses on developing strategic Al capabilities within the controlled environment of Star- Craft II using TextStarCraft II, which poses no inherent risk of dual-use or misuse. Thus, no specific safeguards beyond standard ethical research practices were required.
H.12 Licenses for existing assets
Answer: [Yes]
Justification: All used assets are properly credited with detailed references and URLs. We explicitly state each asset's license type, predominantly CC-BY 4.0, ensuring all terms of use respected and documented as per compliance standards.
H.13 New Assets
Answer: [Yes]
Justification: We introduced TextStarCraft II, new, thoroughly documented asset. Documenta- tion including setup, usage, and licensing details is available in publicly accessible, anonymized repository, ensuring transparency and integrity in the review process, https::/anonnnooot 4open. Science/x//LrrgeLaane Language- Models- Slw.yyyyttarraaat11114455311111
md
H.14 Crowdsourcing and Research with Human Subjects
Answer: [Yes]
Justification:
We involved 30 StarCraft II masters and grandmasters, including professional and semi-srofesiional players, in study. For more details the participant engagement and data handling, please see the section referenced A.
H.15 Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Answer: [NO]
Justification::Athougg our study involved human participants, their engagement was restricted to game-related QA and gameplay testing, which low-risk activities typical of gaming studies. This type of interaction generally does not require IRB approval as it does not involve exposure to phys- ical or psychological risks beyond those encountered in normal gameplay. Participants were fully briefed on the nature of the tests and their voluntary participation was secured, aligning with ethical standards for such research.
57


Reviews:
[{"review_id": "y2TceABOuc", "reviewer": "Reviewer_B3oZ", "summary": "This paper introduces TextStarCraft II, a benchmark designed to assess long-term strategic decision-making in the context of playing StarCraft II. It models the game-playing process through pure textual representation and also develops a chain-of-summarization pipeline to aid LLM's decision-making to achieve victory. They evaluate various LLMs under different prompting strategies against the built-in AI mode of StarCraft II. The experimental results indicate that prompting powerful closed-source LLMs (e.g., GPT-4) and fine-tuned open-source LLMs (e.g., QWEN-7B) can perform competitively with the Level 5 (the highest level) built-in AI of StarCraft II.", "questions": "The paper introduces a benchmark for evaluating large language models' (LLMs) long-term strategic decision-making capabilities in the context of playing StarCraft II. This benchmark models gameplay in a purely textual environment and includes the design of a chain-of-summarization pipeline to aid LLMs in playing and winning the game. The topic is interesting and beneficial to the LLM research community. However, I have several concerns, primarily about the overall pipeline setting and the mechanics of how LLMs engage with the game.\n- The objective of optimizing LLMs through prompting or fine-tuning to win the game appears to be inadequately defined. It remains unclear what the specific winning criteria for LLMs are and how these models compete against the built-in AI to meet these criteria.\n- The methodology detailing how LLMs execute actions and observe the environment in each round lacks clarity. According to Figure 1, each round involves the LLM making decisions regarding necessary actions and translating multi-frame observations into text. However, it is not explicitly described what types of actions are executed or how multi-frame observations are represented within the purely textual environment.\n- The experimental results indicate that well-optimized LLMs, with appropriate prompting, can perform competitively with built-in AIs. It would be beneficial to provide deeper insights.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The topic is interesting.\n- The contribution of proposing a long-term strategic decision making benchmark for evaluating LLMs is beneficial to the community.\n- The evaluation is extensive", "weaknesses": "- The objective of how LLMs achieve victory in the game is not clearly defined.\n\n- The process by which LLMs execute actions and make observations within the textual setting is insufficiently explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Apologies for the lack of clarity in our methodology description. \n\n**Q1**: Standard for Achieving Victory\n\nIn our TextStarCraft2 environment, the victory conditions mirror those of a standard StarCraft II 1v1 game, consistent with established AI benchmarks such as AlphaStar, ROA-Star, or DI-Star. To secure a victory, the LLM agent must destroy all of the built-in AI structures. This criterion is a standard setting across various StarCraft II AI competitions, ensuring that our benchmark aligns with common practices in the field.\n\nTo beat the built-in AI, the LLM must engage in a complex series of long-term strategic decisions. This includes resource gathering, constructing buildings, developing technologies, and producing combat units. The LLM needs to outperform the built-in AI over an average game duration of approximately 20 minutes (7000 steps). This process typically involves intricate resource allocation, tactical maneuvering, and unit countering strategies, among other sophisticated decisions. \nIn our study, we observed some interesting behaviors from LLM agent playing a strategy game. LLM agent was controlling one faction (called Protoss) against a built-in computer opponent controlling another faction (called Zerg). Here are two notable scenarios:\n1. Predicting and Preparing for Attacks(Figure 3.b): \n  - The LLM agent scouted the enemy base and noticed a large army.\n  - It predicted an incoming attack and decided to build defensive structures.\n  - About a minute later, when the enemy attacked, the LLM agent was well-prepared with both defensive buildings and its own army.\n2. Adapting Army Composition(Figure 7): \n  - In the first encounter, the LLM agent's initial army (mostly ground units, zealots) struggled against the enemy's forces(roaches and hydras).\n  - LLM agent then decided to change its strategy, producing more flying units to counter the enemy's ground-based army.\n  - This change in tactics allowed LLM agent to barely survive the first attack wave.\n  - By the time the enemy launched a second attack, LLM agent had built up enough flying units to defend its territory effectively.\n\n**Q2**: Action and Observation Mechanisms \n\n**Multi-frame Summarization**: After each K steps, we collect several frames of game state information, each represented as a textual description. In each round of the game:\n\n  1. We represent each frame as the following text information:\n\n    - Resources (e.g., \"Minerals: 500, Vespene Gas: 200\")\n    - Buildings (e.g., \"2 Gateway, 1 Nexus\")\n    - Units (e.g., \"15 Zealots, 5 Carriers\")\n    - In-process activities (e.g., \"Researching Warpgate: 70% complete\")\n    - Visible enemy units and structures\n    - Current research status\n\n  Then, single-frame summarization step exacts important information(like supply, current army composition) for each frame.  \n  After that, multi-frame approach helps capture temporal dynamics and ongoing processes. We then prompt the LLM to summarize these multiple textual representations into a concise, structured format. This process is designed to mimic how human players process and prioritize game information.\n\n  2. Summarization Process: The LLM is instructed to summarize the multi-frame information in the following structured manner: \n\n    1. Game Overview: A snapshot of key metrics (e.g., game time, worker count, resources, supply).\n    2. Current Game Stage: An assessment of the game's progression. \n    3. Our Situation: \n      - Units and Buildings: A summary of our army composition and structures.\n      - Economy: An evaluation of our economic status.\n      - Technology: An overview of our technological advancements. \n    4. Our Strategy: An interpretation of our current strategic position. \n    5. Enemy's Strategy: An analysis of the opponent's apparent strategy based on observable information. \n    6. Key Information: Highlighting crucial elements that require immediate attention or action.\n\n**Action Execution**: The LLM receives a textual description of the current game state and is prompted to decide on an action. It then outputs a text-based command, which our system interprets and executes in the game environment. \nThe types of actions that can be executed include: \n\n  - Building construction (e.g., \"Build Cybernetics Core\")\n  - Unit production (e.g., \"Train 5 Probes(protoss worker)\")\n  - Research (e.g., \"Research Blink Tech\")\n  - Tactical maneuvers (e.g., \"Scout with 1 probe\", \"Attack enemy base\")\n\nFor example, a command might be \"Build 1 pylon(provide more supply)\" or \"Attack enemy base\".\n\nFor more detailed examples of these textual interactions, please refer to Appendix F: Policy Interpretability Examples.\n\n**Q3**: Deeper Insights                                            \n\nIn our experiments, we observed that optimizing LLMs through strategic prompting is essential for competitive performance against built-in AI opponents. Here are some crucial insights:\n\nPrompt Engineering: Effective prompts blend game mechanics with strategic concepts. For instance, prompts like \"Consider resource management, tech tree progression, and army composition\" led to better results than simpler directives. Summarized game state information at each decision point also enhanced performance.\n\nPrompt Sensitivity: LLMs showed high sensitivity to how decision scenarios are framed. Proactive prompts, such as \"What potential threats should you prepare for?\", generally led to better outcomes than reactive prompts.\n\nFor example, in a test, GPT-3.5 with optimized prompts achieved an 84% win rate against the Level 4 built-in AI, significantly higher than the 12.5% win rate with basic prompts, largely due to improved resource management and anticipatory defensive strategies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6MaPzziTv0", "reviewer": "Reviewer_Q8qA", "summary": "The authors introduce TextStarCraft II, a framework for transferring the state of the Real Time Strategy (RTS) Starcraft II (SC2) into text form and Chain of Summarization (CoS), an extension of the traditional Chain of Thought (CoT) style prompting for condensing information and accelerating LLM inference. The authors demonstrate their system is capable of defeating gold-level, human players in SC2 in real time matches.", "questions": "1.) When playing against human players, where any restrictions placed on availability of units or maps?\n\n2.) Do the authors believe the results would generalize across the RTS genre and has there been any investigation into this?\n\n3.) To what extent were the macro and micro actions predefined?", "limitations": "The authors briefly touch on some limitations of their work, however it is unclear whether the topics addressed in the questions were not included due to being outside the scope of the work, or not relevant given some metric or detail the reviewer may have missed.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The authors introduce a novel benchmark environment for LLMs in the form of TextStarCraft II, an environment that allows for a rigorous evaluation of an LLM’s ability to do real-time, complex planning. They further introduce a new prompting method, CoS, to allow for information compression and multi-step inference to combat the otherwise prohibitively slow inference speed of LLMs when it comes to a RTS game. The performance of their approach when playing against the pre-programmed AIs and human-players proves the effectiveness of their approach.", "weaknesses": "A weakness of the authors’ approach is that there is a high likelihood that most modern LLMs were trained on text that included comprehensive strategy discussions of SC2 in text form. However, this is mostly mitigated by the authors’ investigation of both open and closed-source LLMs in section 5.1. However, it would still be interesting to see if these results still hold in newer RTS games that are likely to not be found in most LLMs’ training distribution. \n\nA lesser weakness the reviewer would like to suggest to the authors is that it may be worth considering staying away from Starcraft-specific terms in order to reach a wider audience that may be less familiar with the Starcraft series as a whole.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. Below is our response to your inquiry.\n\n **Weaknesses**\n\nWe appreciate your suggestion about Starcraft-specific terminology. In our revised version, we will strive to make our paper more accessible to a broader audience by clarifying or simplifying Starcraft-specific terms where possible.\n\n\n**Q1**:  Restrictions in Human vs. LLM Agent Matches\n\nIn our experiments, we maintained a full-range StarCraft II environment with no restrictions on maps, units, or game versions. We utilized 14 different maps from both the 2022 and 2023 ladder seasons and tested across two game versions (5.0.11 and 5.0.12). Human players engaged with LLM agents using standard input devices (keyboard and mouse) under typical 1v1 game settings, similar to the well-known AlphaStar vs. MaNa show matches.\n\nAt the same time, to ensure fairness in the game, we make sure that the LLM agent has access to the same game information as humans, for example:\n\n1. Map Visibility: The LLM agent operates under the same fog of war constraints as human players. It does not have access to any information that would be hidden from a human player in a standard game, ensuring equal information availability for both sides.\n2. Decision Frequency: We calibrated the LLM agent's decision-making frequency to be comparable to that of human players. This means the AI is not making decisions at a superhuman rate, but rather at intervals similar to what a skilled human player might achieve.\n\nThese limits ensure our AI competes fairly with humans. The focus is on testing strategic thinking and decision-making, not mechanical advantages.\n\n\n**Q2**: Generalization Across the RTS Genre \n\n\nWe agree that this is an interesting and valuable observation. Testing LLM capabilities on newer RTS games not included in their training data would indeed provide a fairer assessment and better reflect the models' adaptability to out-of-distribution tasks. While creating a benchmark framework for newer RTS games is more challenging, we are actively working on extending our framework to more complex, recent RTS environments(Such as Storm Gate and Battle ACE). We look forward to sharing these results in future work and welcome your continued interest in this direction.\n\n**Q3**: Predefined Macro and Micro Actions\n\n\n1. We predefine macro actions as follows:\n- Building Construction: The LLM determines which buildings to construct based on its strategic analysis and current game state. This decision-making process follows established community conventions known as \"build orders,\" which can be found at https://lotv.spawningtool.com/.\n- Unit Production: The LLM decides which units to produce, demonstrating its understanding of army composition and resource management.\n- Research and Upgrades: The LLM makes decisions on which technologies to research and upgrades to pursue, aligning with its overall strategy.\n- High-level Military Strategy: The LLM determines when to scout, defend, or launch attacks based on its assessment of the game situation.\n2. Micro Actions and Execution: While the LLM focuses on high-level decision-making, rule-based scripts handle the execution details, for example:\n- Building Placement: Once the LLM decides to construct a building, rule-based scripts interact with the environment to execute the construction.\n- Unit Training Facilities: After the LLM decides which units to produce, scripts select the appropriate facilities to carry out these orders.\n\nBy separating strategic decisions from tactical execution, our framework mirrors real-world scenarios where high-level planning is distinct from operational implementation.\n\n**Limitation**\n\nRegarding the limitations of our work, we will expand and clarify the following points in our revised version:\n\n1. Generality: We will discuss how our methods can be applied to other RTS games and mention that we are extending our framework to more complex games such as \"Storm Gate\" and \"Battle ACE\" to test the adaptability of LLMs.\n\n2. Decision and Execution Balance: We will further explain the balance between LLM decision-making and rule-based execution, describing the separation of macro decisions (e.g., building selection, unit production) from micro execution (e.g., building placement, unit training) and their effectiveness in complex strategic environments.\n\n3. Resource Limitations: We will describe the impact of resource limitations on system performance and introduce our plans to optimize resource usage, including exploring more efficient computing resources to improve overall performance and real-time decision-making capabilities.\n\nThank you again for your insightful comments and feedback, which are crucial for improving the quality of our research. We look forward to continuing this dialogue and further enhancing the contributions of our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8Py25e2dBI", "reviewer": "Reviewer_gmAe", "summary": "This paper introduces TextStarCraft II, a text-based environment to evaluate the strategic decision-making and planning capabilities of large language models (LLMs) in real-time scenarios within StarCraft II (SC2). The study addresses the limitations of traditional Chain of Thought (CoT) methods by proposing the Chain of Summarization (CoS) method, which enhances LLMs’ abilities to process complex information and make strategic decisions efficiently. Key experiments include testing various LLMs against SC2's built-in AI, evaluating commercial models on SC2 knowledge, and conducting human-AI matches to assess performance and strategic adaptability.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Chain of Summarization (CoS) Method: The introduction of the CoS method improves LLMs' ability to summarize and process complex game states, leading to more effective decision-making in real-time scenarios.\n\n- Impact on AI Research: The study’s findings contribute to the broader field of AI research by providing insights into the capabilities and limitations of LLMs in handling complex, real-time decision-making tasks.", "weaknesses": "- Human Interaction Assurance: The paper does not clearly explain how LLMs interact with the game. From lines 120-121, it seems that human players might use LLM suggestions to operate the game. If this is the case, how do you ensure that players fully adhere to the LLM's suggestions?\n- Latency and Real-Time Feedback: In Section 5.3, detailed information on latency is not provided. Can your method provide real-time feedback?\n- Resource Dependency: The reliance on rule-based scripts for micro-management and the limitation to text-based inputs may restrict the diversity and applicability of AI strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\n\nThank you for your thorough review and insightful comments. Below, we address your specific concerns and outline the changes we plan to make in the revised version.\n\n**Q1**:Human Interaction Assurance\n\nWe apologize for any confusion caused by our unclear explanation. To clarify, in our TextStarCraft2 environment, the LLM agent interacts directly with the game environment without human intervention. The LLM receives text-based observations of the game state and outputs text-based decisions, which are then automatically translated into game actions by our system. There is no human intermediary executing LLM decisions. In contrast, LLM follows its own suggestions to decides which action to do (such as \"Build Cybernetics Core\",\"Attack enemy base\"). \n\nWe will revise this section in our paper to provide a more detailed explanation of how LLMs interact with the game environment. Thank you for pointing out the clarity issue.\n\n**Q2**:Latency and Real-Time Feedback\n\nThe latency in our system is primarily determined by the inference speed and model size of the LLM. By leveraging fine-tuned open-source large language models (LLMs), our approach can achieve real-time feedback. Our Chain of Summarization (CoS) method is crucial in enabling this real-time feedback, significantly outperforming traditional Chain of Thought (CoT) methods in terms of responsiveness.\n\nFor instance, with CoS, even smaller LLMs like the fine-tuned Qwen-2 7B can achieve real-time interaction (100 ms/step) and demonstrate competitive performance against human players. This level of real-time capability is not possible with standard CoT methods, highlighting the effectiveness of our CoS approach in facilitating swift strategic decision-making in complex, dynamic environments.\n\nAlthough larger models such as GPT-4, Claude, and Gemini still face challenges in achieving real-time feedback (1s/step) due to their size, complexity, and network transmission factors, our CoS method significantly reduces their response times compared to CoT. This improvement allows for near-real-time performance in many scenarios. We are continuously optimizing our system to further enhance the real-time capabilities of these larger models.\n\nBelow are the delays observed for each step when different LLMs interact with the CoS method: \n | Model Type     | Model Name   | Delay (each step) |\n|----------------|--------------|-------------------|\n| Finetuned Open-Source LLM | Qwen2-7b      | 98ms             |\n| Finetuned Open-Source LLM | Qwen2-1.8b    | 64ms             |\n| Finetuned Open-Source LLM | LLAMA2-7B    | 102ms            |\n| Closed-LLM     | GPT3.5       | 1.2s             |\n| Closed-LLM     | GPT4         | 2.3s             |\n| Closed-LLM     | Gemini-PRO   | 0.3s             |\n\n**Q3**:Resource Dependency\n\nOur LLM utilizes macro actions such as training units, constructing buildings, and researching technologies—commonly known in the StarCraft II community as \"build orders.\" While rule-based scripts handle some micro-level actions, they do not constrain strategic or tactical variety.\n\nIn our experiments, LLM agents have successfully executed diverse strategies. For example:\n\n- Mass Void Rays: A strategy focusing on producing a large number of powerful air units.\n- Stalker-Colossus: A balanced army composition combining ground and support units.\n- Carriers: A high-tech strategy centered around powerful capital ships.\n\nThese examples demonstrate the LLM's ability to adapt unit compositions and respond to potential threats dynamically, showcasing a wide range of strategic approaches without limiting the AI's ability to innovate.\n\nRegarding the limitation to text-based inputs, we consider that focusing solely on text allows us to better assess the core capabilities of language models without the need to consider their abilities in processing other modalities like vision or speech. By expressing the strategic decision-making process through text interactions, we can concentrate more on evaluating the language models' strategic reasoning and planning abilities in complex and dynamic environments.\n\nOn the other hand, we also recognize the potential benefits of incorporating visual information. We are actively exploring the integration of vision-language models (VLMs) into our framework. This multimodal approach could enhance the model's overall ability to process information and make AI decisions, potentially leading to more sophisticated and adaptable strategies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers and I agree that this paper makes a solid contribution to benchmarking for LLMs. While the paper is light on theoretical contributions, it is an excellent piece of empirical work and will likely be of interest to a wide variety of researchers.

As a side note, I sincerely hope you're working on trying this with visual data using a multi-modal LLM!
Decision: Accept (poster)