Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning
Authors: Daechul Ahn
San Kim
Jonghyun Choi
Venue: COLM 2025 (2025)
Keywords: multi agent
real-time simulation
strategic reasoning
TL;DR: We propose a multi-agent framework uniting specialized imitation learning modules under a meta-controller, achieving robust long-horizon strategic reasoning and superior adaptability in dynamic environments.

Abstract: Large Language Models (LLMs) have recently demonstrated impressive action sequence prediction capabilities but often struggle with dynamic, long-horizon tasks such as real-time strategic games. In a game such as StarCraft II (SC2), agents need to manage resource constraints and adapt to evolving battlefield situations in a partially observable environment. This often overwhelms exisiting LLM-based approaches. To address these challenges, we propose a hierarchical multi-agent framework that employs specialized imitation learning agents under a meta-controller called Strategic Planner (SP). By expert demonstrations, each specialized agent learns a distinctive strategy, such as aerial support or defensive maneuvers, and produces coherent, structured multistep action sequences. The SP then orchestrates these proposals into a single, environmentally adaptive plan that ensures local decisions aligning with long-term strategies. We call this HIMA (Hierarchical Imitation Multi-Agent). We also present TEXTSCII-ALL, a comprehensive SC2 testbed that encompasses all race match combinations in SC2. Our empirical results show that HIMA outperforms state of the arts in strategic clarity, adaptability, and computational efficiency, underscoring the potential of combining specialized imitation modules with meta-level orchestration to develop more robust, general-purpose AI agents.

Published as a conference paper at COLM 2025
Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning
Daechul Ahn; San Kim", Jonghyun Choi
Seoul National University
[daechulahn, 00sankim, jongguuchhoo @enuui @enuu
Abstract
Large Language Models (LLMs) have recently demonstrated impressive ac- tion sequence prediction capabilities but often struggle with dynamic, long- horizon tasks such as real-time strategic games. In game such as StarCraft II (SC2), agents need to manage resource constraints and adapt to evoly- ing battlefield situations in partially observable environment. This often overwhelms exisiting LLM-based approaches. To address these challenges, we propose a hierarchical multi-agent framework that employs specialized imitation learning agents under meta-controlla called Strategic Planner (SP). By expert demonstrations, each specialized agent learns a distinctive strategy, such as aerial support defensive maneuvers, and produces coherent, structured multistep action sequences. The SP then orchestrates these proposals into single, environmentally adaptive plan that ensures local decisions aligning with long-term strategies. We call this HIMA (Hierarchical Imitation Multi-Agent). We also present TEXISCII-ALL, a comprehensive SC2 testbed that encompasses all race match combinations in SC2. Our empirical results show that HIMA outperforms state of the arts in strategic clarity, adaptability, and computational efficiency, underscoring the potential combining specialized imitation modules with meta-level or- chestration to develop more robust, general-purpose Al agents. We release our code, model and datasets at https: / thub. com/ http//snmpllbbbbba /hima.
1 Introduction
Large Language Models (LLMs) have achieved remarkable success in various tasks (Brown et al., 2020; Raffel et al., 2020; Ouyang et al., 2022; Touvron et al., 2023) but its ability to understand complex, ever-changing environments to reason sequence of actions remains challenging (Brohan et al., 2022; Driess et al., 2023; Fan et al., 2022). Generating proper action sequences in the changing environment requires strategic reasoning to address uncertainty of the changing environment (Gandhi et al., 2023). We argue that real-time strategy (RTS) games exemplify these challenges. In StarCraft II (SC2) (Blizzard Entertainment, 2010), for instance, players must gather resources, build infrastructure, manage expansions, and produce units simultaneously in real time, while operating under partial observability of the battlefield. This interplay of short-term tactics (egg,, efficient early builds) and non-trivial objectives (e.g., achieving aerial dominance over the battlefield) makes SC2 a well-suited domain to test strategic reasoning (Vinyals et al., 2019).
Recently, Ma et al. (2024) propose TextStarCraft II, text-based evaluation environment for SC2 where game states and actions processed text, featuring multiple controllable units, diverse resource requirements, and restricted visibility in an evolving battlefield (see details in Appendix Sec. B). Existing L.LM-based approaches (Ma et al., 2024; Shao et al., 2024; Wu & Hu, 2025; Li et al., 2025b) can interpret game states to predict a sequence of actions, but they often struggle to maintain efficient resource management and coherent build orders,
'These authors contributed equally.
*IC with ECE, ASRI and IPAI in SNU and corresponding author.
1

---

Published as a conference paper at COLM 2025
Frequent Short-Horizon Actions:
Structured Long-Horizon Plan
Exacutable LLM Calls
(TexStarCratt II Ma el 2024)
(HIMA-OURe
Non-Execu_lblle LLM Calls
Decision at time 00:00
Decision time 00:00
Win rate at Dift. Lv.7 PP Z)
<TRAIN PROBE>
<TRAIN PROBE>
<BUILD GATEWAY>
<TRAIN PROBE>
<TRAIN PROBE>
<BUILD PYLON>
<TRAIN PROBE>
600
86
Win rate (%) at Game Diff. Lv.
<BUILD PYLON>
<TRAIN PROBE>
Number of LUM Ca
500
<TRAIN PROBE>
<BUILD GATEWAY
60
Decision at time 00:01
400
<TRAIN PROBE>
<TRAIN ZELOT>
Decision at time 00:42
300
40
<TRAIN ADEPT>
<BUILD PYLON>
<TRAIN PROBE>
<TRAIN PROBE
<BUILD PYLON>
200
<BUILD C-CORE>
<TRAIN PROBE>
20
<TRAIN PROBE
100
Decision at time 00:02
<BUILD GATEWAY>
TextStarCrat HEP
HIMA (Ours)
V: Feasible action Infeasible due to unmet prerequisie
(Mattal (Uetta 202)
(a) Difference of action planning paradigm
(b) LLM calls and win rate
Figure 1: Comparison of short-horizon (Ma et al., 2024) us. proposed long-horizon (HIMA) planning in SC2. Fewer LLM calls, more coherent build orders, and higher win rates in HIMA. (a) Existing methods frequently produce one-step actions at short intervals (eg., 00:00, 00:01, 00:02,....,, causing repeated invalid actions (X) due to unmet prerequisites. In contrast, HIMA produces structured multi-step plans with fewer queries (e.g., 00:00, 00:42), respecting build orders and maintaining coherent strategy. (b) Quantitative results showing LLM calls over average of 20 minutes of gameplay (blue: executable; red: non-executable) and win rate against level 7 AI (Protoss us. Zerg: green line). HIMA requires fewer LLM calls while achieving higher win rates, demonstrating superior efficiency and effectiveness.
egg, producing actions that ignore required prerequisites or repeatedly producing redundant commands, wasting resources, and disrupting build orders, as shown Fig. 1.
One way to mitigate these issues is to use imitation learning (IL) (Abbeel & Ng, 2004; Torabi et al., 2018; Urakami et al., 2024; Wu et al., 2025), which uses human demonstrations to capture the core patterns of expert play. Although II. can impart valuable strategy to model, a purely imitation-driven approach may fail when faced with novel or evolving battlefield scenarios in which the opponent's 's capabilities differ from those seen in the training data. Furthermore, in RIS games such as SC2, multiple paths to victory exist - such as early aggressive rushes, aerial dominance, to adapt to diverse strategic routes in each situation. This breadth of strategies, each with distinct requirements and tactics, often poses challenges for single monolithic model.
balanced resource management require agents
Another way is to use reinforcement learning (RL) (Vinyals et al., 2019). However, it requires much computation for many trials and errors, which may not justify the cost versus efficacy trade-off, and a well-designed reward function, which is often not trivial.
To address these limitations in cost-effective manner, draw inspiration from the 'society of mind' principle (Minsky, 1986) and propose a multi-agent framework that coordinates specialized imitation learning agents under a central meta-controller (called the Stralegic Planner (SP)), which we collectively refer to as HIMA (Hierarchical Imitation Ageni-Agent), as shown in Fig. 2. Each specialized agent generates longer-horizon structured action se- quences (Sec. 3.1), ensuring that procedural prerequisites are fulfilled and reducing invalid or redundant commands (Fig. 1). Before committing to a final decision, the SP then synthesizes these proposals through an environment-quue -woare action orchestration, using proposed 'temporal Chain-of- -haouhhf-Thought (t-CoT)' reasoning. induces the SP to align the final decision with immediate, short-term, and long-term objectives (Sec. 3.2). In particular, the SP continu- ously monitors the changing battlefield conditions, adapting its strategic decision based changing environments before finalizing action plans. Using longer-horizon structured action sequences from imitation agents, the SP requires significantly fewer adjustments than the prior arts that frequently query LLMSs at every time step, as illustrated in Fig. 1.
2

---

Published as a conference paper at COLM 2025
Player state
Current resources
Armybuildingg/rssercch state
Pending actions
Imitation multi-agent for structured action sequence
Transtionn strong air composition with Carries and Voldrays, while expanding your economy building...
Focus training Probes to reach healthy economy while building your Storgate and Fleeibeacon for
Build onditional Gataways and Pylons Increase production capacity, while also training Adepts and Zealot for
Player state
Player state
Player state
Air Superiority Specialist
Infrastructure Specialist
Superiority Specialist
Environment
Strategic planner
Player state
Final decision
Opponent state
<BUILD SHIELDBATTERY- x <TRAIN VOIDRAY> 3
Action outcome
<TRAIN PROBE> <BUILD STARGATE>
Map information
<BUILD FLEETBEACON: <TRAIN ADEPTs
Figure 2: Overview of the proposed hierarchical imitation multi-agent (HIMA) frame- work. Each specialized imitation agent (egg,, Air Superiority, Infrastructure, Ground Superi- ority) receives up- up-to-date player state information (resources, units, buildings) and produces a multi-step action plan along with a rationale explaining its strategic intent. A high-level meta-controller (i.e., Strategic Planner) then merges these agent-level proposals into single cohesive decision, factoring in broader environmental contexts (opponent state, action out- comes, battle progress). This hierarchical structure enables long-horizon planning, adaptive coordination among specialized agents, and real-time responsiveness to evolving battlefield.
SC2 Evaluation Environment
Player Race(s)
Opponent Race(s)
#Matchups
TextStarCraft II (Ma et al., 2024) SwarmBrain (Shao et al., 2024)
Protoss
Zerg
1
Zerg
Terran
TEXTSCII-ALL (Ours)
Protoss, Zerg, Terran
Protoss, Zerg, Terran
9
Table 1: Comparison of text-based SC2 evaluation environment. Unlike previous work with single player race and limited enemy matchups (one each), our TEXTSČII-ALL coverss all three races on both sides, resulting in a total of nine possible match-ups.
In addition, to evaluate SC2 comprehensively, we present an expanded SC2 evaluation envi- ronment that encompasses all three races (Protoss, Terran and Zerg) and their nine possible match-ups, as shown in Tab. 1, which we call TEXISCII-ALL. Beyond single match-up stud- ies in the literature (Ma et al., 2024), new environment exhibits comprehensive assessment of matches all race combinations. We empirically validate methods in this comprehensive environment, demonstrating that HIMA outperforms sate-of-the-arte (SoTAs) in win rate against built-in AI, head-to-head matches, and computational efficiency.
We summarize contributions as follows: (1) we propose hierarchical imitation multi-agent framework that enables efficient and effective long-term strategic planning in SC2; (2) we expand the SC2 evaluation environment to include all nine race matchups, offering more comprehensive benchmarking testbed; and (3) we demonstrate the effectiveness of our proposed HIMA against both built-in Al opponents and direct match-ups with SoTAs.
2 Related Work
Interactive environment. Real-time strategic decision-making environments have progres- sively increased in complexity. Tasks have expanded within physics-based simulations and gaming platforms, evolving from simpler frameworks like MujoCo (Todorov et al., 2012) and ALE (Bellemare et al., 2013) to sophisticated open-ended systems including MineCraft (Wang et al., 2024; Li et al., 2025a), PokeLLMon (Hu et al., 2024), ALFWorld (Shridhar et al., 2021), and ScienceWorld (Wang et al., 2022), among others (Albrecht et al., 2022; Qi et al., 2024). These systems often involve partial observability and tasks mirroring real-world scenarios.
3

---

Published as a conference paper at COLM 2025
Recent LLM-based SC2 testbeds, including TextStarCraft II (Ma et al., 2024) and Swarm- Brain (Shao et al., 2024), present formidable obstacles for language models and autonomous agents. They demand intensive operational control, navigation through expansive action spaces, processing of complex state representations, and long-range planning over extended gameplay sequences, underscoring modern interactive environments' heightened difficulty.
Multi-agent interaction. The Society of Mind theory proposed by Minsky (Minsky, 1986) posits that intelligence materializes through interactions between specialized agents-4 foundation in multi-agent systems (MAS). Within MAS frameworks, autonomous agents en- gage in either collaborative or competitive behaviors to facilitate collective reasoning across numerous domains (Zhang et al., 2019; Chen et al., 2024a; Park et al., 2023; Chen et al., 2023; Li et al., 2023). The advent large language models (LLMs) has catalyzed the development of advanced multi-agent architectures, encompassing methodologies such as Voting (Wang et al., 2023), Debate (Du et al., 2023), Reconcile (Chen et al., 2024b), Role-playing (Tseng et al., 2024; Long et al., 2024), and comprehensive societal simulations (Paquette et al., 2019; Qi et al., 2024). Here, we present HIMA, a multi-agent framework that employs imitation- based interactions coordinated by sophisticated meta-controller for real-time strategic reasoning task such as SC2. This meta-controller adaptively synthesizes environmental contexts with agent-specific objectives, enabling cohesive inter-agent communication that markedly improves strategic coordination and system efficacy.
3 HIMA: A Hierarchical Multi-Agent Imitation Framework
To address the limitations of existing L.L.-based methods such as producing short-horizon actions and invalid build orders, we propose HIMA, a hierarchical multi-agent architecture (Fig. 2) that deploys specialized imitation agents trained on human demonstration data for generating diverse structured action sequences (Sec. 3.1), coordinated by Strategic Planner (SP) that integrates these proposals and aligns them with overarching objectives while considering broader environmental contexts (Sec. 3.2).
3.1 Learning Multi-Agent for Structured Action Sequence
In SC2, the number of actions executed per timestep increases as the match progresses, as observed in the human demonstration (Appendix Sec. H). The early stages are constrained by limited resources and procedural prerequisites, while the later stages enable a diverse action space and require more actions per time step to manage the increasing complexity of units, buildings, and resources an evolving battlefield. Existing approaches that generate fixed number of actions per timestep (Ma et al., 2024; Li et al., 2025b; Wu & Hu, 2025) often fail to keep pace with this evolving "tempo," producing frequently fragmented strategies.
Structured action sequences with rationales. To address the strategy fragmentation, we take inspiration from human gameplay, where players naturally vary their frequency of action as the match progresses, taking into account both immediate strategic conditions and future objectives. Specifically, rather than producing fixed action counts A; per decision point, we design our agents to generate strucfured action sequences that span a time window A, denoted as Am+A This enables variable action density within each window, reflecting gameplay patterns where later phases require more actions in the same timeframe. To guide these variable-density action sequences, we also append Tactical Rationale (TR) explaining why each action set suits its particular game phase and context. This explicit reasoning improves decision quality by providing strategic context (see Appendix Sec. for empirical benefits).
We implement this idea in three steps as follows (illustrated in Fig. 3-(a)). First, we extract state-action pairs {S,, Ar} from the SC2EGSet (Bialecki et al., 2023), dataset containing professional SC2 replays annotated with detailed game states and corresponding player actions (see Fig. 3 (left) and Appendix E.1). Then, we prompt LLM (GPT-40-mini (OpenAI, 2024)) to generate TR for action sequences Apt Finally, we convert the results into the instruction tuning format for supervised fine tuning (SFT). We provide detailed construction
4

---

Published as a conference paper at COLM 2025
Step 1: Extract state- action from SC2EGSet
Step 2: Generate tactical rationale from state and actions
Input: Player state (S.)
Action orchestration
TR&Acioos
TR&AAtores
TR&Actrn
Current assessment
SC2EGSet (aoeokettal 2023)
Economy is slable because..
Question: Justly actires from
Player state (SU)
Advisor strolegy resolutton
& Actions (Him-s)
Basod on thoir TR, A agroe on the importance completing Considered SO, aetial domin critcal, prioiiize
"TypaName": 'nitBom", 'd":0, "lopp:: 11680.
"playerid: 1, "stats":
3 Develop that counter the enemy's s ground foroes
Strategy formulation
"FoodMadee: 56546,
Uszdf 40554,
Tectical Rationale (TR): Aim balancad army, expand economy with Nexus, secure air tach.
Strategic planner
"Unil Pagindggdd 283, "UnilNome" "Zeaiot.
Action planning (+CoT)
Immedaate action: Consider pravious fail command bulld fleetbsacon first to meet car Short-erm actirs: To address upcoming threatfom opponent...
LLM
Opponent state
Step 3: Add rationale to dltta
Env.
We attack!!
Langtorm torm actire: keep up wit astial tech advancement...
The has Roach,
Instruction-tun dataset
Ravager Mutalisk
Final decision
Player state (8:) &
Input: Player (5g
Output TR Actios
Your prev response <TRAN
Action outcome
<BULD 5:3AT TERY> <TRAIN CAHRERR <RESEARCH AIR ARMORS
BEACONN <TRAIN VOIDRAYS
Action (A,,)
(Actoonato for specific Ime-window, A.t.a)
unmet preroquistes
(a) Dataset for structured action sequence
(b) Environment-aware action orchestration
Figure 3: Dataset construction pipeline for generating structured action sequence and environment-aware action orchestration in HIMA. (a) We extract state-action pairs {S,, Ar) from SC2EGSet (Bialecki et al., 2023) and prompt LLM to generate Tacti- cal Rationale (TR) for each multi-step action sequence, Agt+A- This TR is appended to form an instruction-tuning dataset, capturing what actions occur and why they re chosen. (b) In deployment, multiple specialized agents (A, ...,, K) produce pairs of Stralegic Objective (SO) and TR, which the strategic planner reconciles based on current player and opponent states. This enviromeent-wware process ensures adaptive short-term responses miti- gating failed commands-ane and consistent long-term planning (egg,, securing air superiority) by integrating rationales and outcome feedback from ongoing game events.
procedures in Appendix Sec. E.1. This approach mirrors human gameplay, where the frequency of action increases with the complexity of the game.
Multi-agent specialization. Even with structured action sequences, covering the vast strategic space of SC2 remains a challenge for single model. For instance, one strategy might focus on quick air dominance (using mass flying units), while another aims at well-fortified ground army. Thus, we adopt multi-agent approach (Wang et al., 2023; Du et al., 2023; Chen et al., 2024b; Long et al., 2024), where each agent specializes in particular 'strategic persona'. Specifically, we leverage the instruction-tuned data and group the agents by unit compositions (egg,, ground-heavy vs. air-heavy armies) which a crucial factor influencing build orders and combat styles in SC2 using k-means clustering. Since similar compositions often indicate consistent strategic patterns (e.g., mass air vs. strong ground), each cluster produces specialized agent. Combining these agents covers wider range of strategies than a single model. Subsequently, a meta-controller (Sec. 3.2) merges their action sequences into a unified one that adapts real-time developments.
3.2 Strategic Planner
In HIMA, the SP acts meta-controller that fuses structured action sequences from specialized imitation agents into a coherent long-range plan, as illustrated in Fig. 3-(b).
Environment-aware action orchestration. We plan actions in four stages as follows.
1. Current assessment:
The SP examines the latest game state (e.g., resources, unit compo- sitions, visible enemy units) to establish current context.
2. Advisor strategy resolution: The SP then applies the Nominal Group Technique (NGT) (Delbecq & de Ven, 1971; Long et al., 2024), a structured decision-making method, to systemically analyze multiple agents' strategic proposals, as shown in Fig. 3-(b).
5

---

Published as a conference paper at COLM 2025
Specifically, the SP applies structured four-step process: (i) identifying agreed and conflicted viewpoints among agents; (ii) resolving conflicts by evaluating each agent's rationale; (iii) considering isolated but insightful viewpoints; and (iv) synthesizing these inputs into a cohesive final strategy. In particular, the SP resolves the strategies by explic- itly considering the intentions behind each action proposed by the agents. To achieve this, the SP evaluates each agent's i TR (justification for the proposed actions) along with its corresponding Strategic Objective (SO), high-level strategic goal derived from expert gameplay analysis (see Appendix Sec. K for details and Fig, 10 for examples).
3. Strategy formulation:
The SP balances each agent's logic, goals, and battlefield con- straints to produce consolidated action plan.
4. Temporal Chain-of-Thought Finally, the SP breaks down the chosen strategy into immediate actions (urgent responses), short-term actions (near-term objectives), and long- term actions (larger strategic goals) before generating a final decision. We refer to this systematic alignment of different time horizons as temporal Maain-of-Thoughh (t-CoT) as connects immediate, short-term, and long-term goals into a step-by-step reasoning chain that reflects how players plan over time.
By the four stages, the SP effectively orchestrates the specialized agents' actions, maintaining coherent decision-making in an evolving game state (see more detailed prompts in Figures 14 and 15). Furthermore, since each imitation agent produces a structured action sequence over fixed time window A the SP requires significantly fewer plan updates compared to previous methods (Ma et al., 2024; Wu & Hu, 2025; Li et al., 2025b) that query LLMs at frequent fixed intervals, substantially reducing LLM call overhead while maintaining strategic depth, as illustrated in Fig. 1-(b).
Feedback system for immediate adaptation. Although long-term planning through structured action sequence is beneficial to efficiently building coherent strategies, can limit responsiveness if the state of the game changes unexpectedly. To address this, the SP employs a feedback system that reexamines the current plan after critical events. a sequence fails to meet its objective, the SP records the failure and refines subsequent prompts to prevent repeating similar errors, as illustrated in Fig. 3-(b). Likewise, if the agent detects an unexpectedly large enemy force (iee,, when the number of enemy units exceeds a threshold, r=10, which is heuristically determined), the SP discards the current plan and requests new action sequence starting from imitation agents. This mechanism blends proactive long-range planning with the real-time adaptability needed to handle sudden changes in RTS gameplay. More comprehensive scenarios demonstrating the feedback system's operation are available in Appendix Sec. O.
By integrating specialized agents' proposals through environment-aware action orchestra- tion and adapting through the feedback system, the SP generates a strategy that responds to changing battlefield conditions.
4 Experiments
4.1 Experimental Setup
Training and inference. We primarily use Qwen-2 1.5B (Yang et al., 2024) as our base imitation learning agent, with additional open-source models described in Appendix Sec. L. For SFT, we extract 39.9k, 39.8k, and 30.8k training instances for Protoss, Zerg, and Terran from human demonstrations (Biatecki et al., 2023), converting them to instruction-tuning format. We set A as 3 minutes for structured action sequence based on the experiments (Fig. 4-(c) and k = 3 groups per race validated through empirical testing (Appendix Sec. G) Details about instruction-tuning dataset construction are in Appendix Sec. E. During inference, we primarily employ GPT-4o-mini (OpenAI, 2024) the SP, with additional evaluations on other open-source and closed-source models described in Appendix Sec. M. Unless otherwise specified, we measure win-rates over 50 randomly sampled matches.

---

Published as a conference paper at COLM 2025
Win Rate (%) at Game Difficulty
Method
Player us. Opponent
Lv.4
Lv.5
Lv.6
Lv.7
Lv.8
Lv.9
Lv.10
lextStarCraft (Ma al., 2024)
Protoss vs. Zerg
84
SwarmBrain (Shao et al,
Zeng Terran
100
76
EpicStar (Wa & Hu, 2025)
Protoss us Zerg
67
30
HEP ali,
Protoss øs Zerg
100
75
75
25
Protoss vs. Protoss
90
80
68
48
42
14
4
Protoss ns. Terran
94
86
70
68
48
24
8
Protoss Zerg
100
92
84
82
68
20
16
Zeng us. Protoss
70
48
8
0
0
HIMA (Ours)
Zerg us. Terran
100
84
12
8
0
0
Zeng us. Zerg
72
50
12
14
0
Terran us. Protoss
58
28
16
10
0
0
Terran us. Terran
62
4
0
0
0
0
Terran us. Zerg
60
32
6
0
Table 2: Win rates (%) across all nine race matchups (Lv.4-Lv.00) our TEXTSCHI-ALL evaluation environment and performance of SoTAs. Rows sharing the same color shading indicate the same Player us. Opponent matchup. Whereas prior baselines (Shao et al., 2024; Ma et al., 2024; Wu & Hu, 2025; Li et al., 2025b) each focus a single matchup, we evaluate pairs (Protoss, Terran, Zerg). Each row reports win rates over 50 games per difficulty level; dashes (-) denote untested conditions. Our Terran results are comparatively lower, likely due to its higher micro demands (we primarily target macro control), which we plan to address in future work.
HIMA on all playe--versus-pponee
TEXTSCII-ALL evaluation environment. We propose new text-based StarCraft II eval- uation environment, called 'TEXTSCIIAALI' which expands upon the existing TextStar- Craft II (Ma et al., 2024) environment. While previous work only supported single matchup-Protoss (player) vs. Zerg (opponent) our TEXISCII-ALL evaluation environ- ment includes comprehensive battles across all three races (Protoss, Terran, Zerg) and supports all nine possible player-opponent race combinations. To achieve this, define and implement complete, race-specific action spaces for Terran and Zerg in addition to Pro- toss, enabling any race to serve as either the player. This broader action space and enhanced evaluation scope facilitate comprehensive and fair benchmarking. Implementation details and the full definition of each race-specific action space are provided in Appendix Sec. D.
Evaluation metric. Ma al. (2024) propose various evaluation metrics, including win-rate, to assess model performance. However, many of these metrics do not closely correlate with the primary measure of success, namely, the win rate, especially as the game progresses (but provide further details these metrics and results in Appendix Sec. F). Consequently, we employ win-rate as our main evaluation criterion under different difficulty levels.
4.2 Baselines
We compare our proposed HIMA against recent SoTA approaches: SwarmBrain (Shao et al., 2024), TextStarCraft (Ma et al., 2024), EpicStar (Wu & Hu, 2025), and the very recently proposed HEP (Li et al., 2025b) in both built-in AT and head-to-head matchups.
4.3 Match-up Results
With built-in AI. We match up HIMA with built-in Al system in all nine race com- binations (with Protoss, Terran and Zerg) for the first time in the literature (cf. prior arts focus single race combination) and summarize the results in Table 2. In all match-ups, our approach exhibits high win rates over multiple difficulty levels. In particular, while HIMA consistently achieves high win rates in Protoss and Zerg matches, its performance is
7

---

Published as a conference paper at COLM 2025
Opponent
Ours vs. Opponent
Win Rate (%)
SwarmBrain al., 2024)
Protoss vs. Zerg
100
TexiStarCraft (Ma et al,, 2024)
Protoss us. Protoss
100
HIEP (Li 2025b)
Protoss US. Protoss
100
Table 3: Match-ups between HIMA (Ours) and SoTAs. All results use HIMA (Protoss) against the listed opponents, measuring the win rate over 10 consecutive games. Our method achieves a 100% win rate, outperforming all previous approaches.
Win Rate (%) at Game Difficulty
Agent Configuration
IL Agent Size
Lv.4
Lv.5
Lv.6
Lv.7
Lv.8
Lv:9
Lv.10
Single-Agent (IL Only) (Ma et at.
7B
70
25
0
0
0
0
Single-Agent (IL Only)
7B
90
75
25
15
0
0
Single-Agent (IL. + SPP)
7B
90
80
35
20
10
0
0
Multi-Agent (IL SP) (Ours)
1.5B x
100
92
84
82
68
20
16
Table 4: Ablation study of different agent configurations. We compare four agent con- figurations across difficulty levels 4-10: Single-Agent (IL Only) (Ma et al., 2024)trained previously provided instruction-tuninn
data from Ma et al. (2024).; Single-Agent (IL Only) trained on our human demonstrations; Single-Agent (II. SP) with Strategic Planner added; and Multi-Agent (IL SP) using three 1.5B agents (total ~4.5B). Results show our dataset improves low-level performance (first row second row), adding SP enhances mid-level success (third row), and the multi-agent framework maintains robustness at higher diffi- culties (fourth row). Each in the first three reports shows win rates over 20 games per difficulty level, while the fourth shows rates over 50 games.
lower for Terran. We conjecture that this result stems from the need for more fine-grained microlevel control, which goes beyond the macro-focused scope of our current evaluation environment. Addressing these scenarios remains a key direction for future work.
With state of the arts. We match our HIMA with three open source baselines and sum- marize the results in Table 3. Unfortunately, we cannot compare EpicStar (Wu & Hu, 2025) with ours because it has not yet provided publicly available code. For each opponent, we use the same race setting originally reported to achieve their best performance. We select Protoss for HIMA our previous experiments with built-in Al showed that HIMA (Protoss) consistently achieved the highest win rate among all opposing races. We measure performance in 10 consecutive games in each match, showing that HIMA A achieves 100% win rate in all cases, showing the effectiveness our proposed method.
4.4 Detailed Analysis
For a detailed analysis, we consider only Protoss (player) vs. Zerg (opponent) because it represents one of the most commonly studied matchups in previous work (Ma al., 2024; Li et al., 2025b; Wu & Hu, 2025) by running 20 randomly sampled matches.
Agent configurations. We compare four setups from Lv.4 to Lv.10 and summarize the results in Table as follows. (1) Single-Agent (IL Only) (Ma al., 2024) uses previously available instruction-tuniig data from Ma et al. (2024); (2) Single-Agent (IL Only) leverages our human demonstration-basee dataset, which notably improves performance at lower difficulties; (3) Single-Agent (IL + SP) adds Strategic Planner (SP), which increases mid-level win rates; and Mullti-Agettaa (IL + SP) employs three 1.5B agents (total 44.BB) comparable to 7B single agent. Results show that while the single-agent models falter beyond Lv.7, the multi-agent approach remains more resilient, sustaining higher win rates at Lv.8100
Multi-agent clustering criteria. We cluster instruction-tuning data by (1) opening strategy, (2) advancement tempo, and (3) unit composition to train agents with distinct characteristics.
8

---

Published as a conference paper at COLM 2025
Win Rate (%) at Game difficulty
Clutering criteria
Lv.4
Lv.5
Lv.6
Lv.7
Lv.8
Lv.9
Lv.10
Opening strategy
0
0
Advancement tempo
40
35
Unit composition (Ours)
100
92
84
82
68
20
16
Table 5: Performance comparison of different multi-agent clustering criteria at various game difficulty levels. We experiment with various clustering criteria and empirically confirm that clustering based unit composition yields superior performance.
Win Rate (%) Game Dift. Lv.7
80
Win Rate (%9 at Game Lv.7
80
Win Rate (%) at Game Lvv77
80
Win API calls
Number of Cals
60
60
00
50
30
30
Simple NGT -R-ST
We 1 w/ :OoT
20
4
Aggregation Method
t-CoT Usage
Planning Time Horizon (min)
(a) Aggregation methods
(b) Necessity t-t
(c) Planning time horizon (A)
Figure 4: Detailed analyses of the proposed HIMA. (a) Enhanced aggregation methods improve performance, (b) t-CoT effectively bridges short- and long-term strategies and (c)
optimal planning horizon balances reactivity and efficiency.
The opening strategy uses public metadata* in early build orders. Advancement tempo differentiates tech-rushing from early-unit production strategies. For the unit composition, we group replays based on the types of unit used each match-up. As shown in Tab. 5, unit composition clustering performs best, guiding the planner to select optimal units for evolving game conditions (Appendix Sec. E.2 for details). Furthermore, we analyze how response diversity generated through these various clustering approaches impacts overall performance in Appendix Sec. I.
Aggregation method of the strategic planner. Figure 4-(a) shows comparative results three aggregation methods for the SP Simple merges action sequences without coordi- nation (i.e,, without advisory strategy resolution in Sec. 3.2). NGT applies Nominal Group Technique (Delbecq de Ven, 1971) transform agents' proposals into a unified plan con- sidering the environment. NGT + TR-SO extends this by incorporating Tactical Rationale and Strategic Objective from each agent. Results show NGT + TR-SO achieves the highest win rate at difficulty level 7, attributable to the synergy of combining tactical justifications with strategic goals, allowing coherent decision-making under dynamic conditions.
Temporal COT. We compare win rates with and without the proposed 'temporal Chain-of- Thought (t-CoTY and summarize the results in Figure 4-(b). Results show t-CoT outperforms the baseline, indicating that systematically linking immediate, short-term, and long-term goals enhances strategic coherence and responsiveness. The performance gap underscores the importance of temporal reasoning hierarchical planning for competitive environments.
Time-window selection for specialized agents. We illustrates how extending the time window (A) for action sequence generation affects both average API calls and win rates (at game difficulty Lv. 7) in Figure 4-(c). With short planning horizon (e-g., minute), the agent queries the model more often and reacts quickly, but this frequent disruption raises computational overhead without necessarily improving performance. As the horizon
httpps: htttp:://lot..spaaiinntoot www..ssaainnntol.ccommm hwwp:/tooooo..oo/p/paeaa.aa.aaae ?pro only=on

---

Published as a conference paper at COLM 2025
Method
Time per LLM call
Avg. #LLM calls (in 20m)
Total LLM call time (s)
TextStarCraft (Ma 2024)
12.1
636
7,695
HEP (l al,, 2025h)
10.4
468
4,867
HIMA (Ours)
22.5
247
Table 6: LLM call overhead across 20 winning matches. Each averaging about 20 minutes. While the hierarchical multi-agent framework of HIMA increases the duration of individual LLM calls, its dramatically lower call frequency substantially reduces overall LLM overhead (e.g,, 247 s vs. several thousand seconds). In real-time play, frequent LLM calls can stall game progression, highlighting the need to minimize total call time.
lengthens, average API calls decline steadily, freeing up resources while still providing enough adaptability for mid-game shifts.
However, beyond certain midpoint (here, around 3-4 minutes), over-long planning hori- zons become too rigid, missing critical opportunities to respond to immediate threats. This trade-off produces peak in the win rate an intermediate horizon, demonstrating that moderate time windows strike the best balance between a timely reaction and long-term cohesive strategy.
LLM call overhead. To compare LLM call overhead, we summarize the average time per LLM call, calls on average 20 minutes, and the total LLM time over 20 matches in Table Despite longer call time (22.5 seconds) in our proposed HIMA due to its three specialized agents and GPT--4-mini planner, HIMA makes only 11 calls per 20-minute game through effective long-term planning. This efficiency reduces the total LLM response time 247s, compared to thousands for other methods. In real-time settings, where game play continues during processing, HIMA's reduced call frequency provides smoother play experience.
5
Conclusion
We propose hierarchical imitation multi-agent (HIMA) framework for SC2, where spe- cialized agents generate structured action sequences and a meta-controller fuses these into cohesive strategies. Using human replay data with tactical rationales, method preserves build orders, reduces LLM queries, and adapts to dynamic conditions. We also introduce the TEXISCII-ALL environment, which covers all nine race matches, comprehensive RTS testbed. The results show that HIMA achieves better win rates and efficiency compared to SoTA baselines, demonstrating the effectivenee of combining imitation-based specialization with high-level orchestration in complex RIS environments.
Acknowledgment
This work was partly supported by CARAI grant funded by DAPA and ADD (UD230017TD, 45%) and the IITP grants No.RR-22022-1222077, No.RS-2022-11220113 13, N0.25-2022-11220959, RRR-2022-11220865 2021-11212068 (Al Innov. Hub), No. RS-2025-25442338 (AI Star Fellowship-SNU) funded by the Korea government(MSIT).
No. RS-2025-02263598 (20%), No.RS-2001--1211135 (SNU AI), No.RS-
References
Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement
learning. In ICML, 2004.
Joshua Albrecht, Abraham Fetterman, Bryden Fogelman, Ellie Kitanidis, Bartosz Wróblewski, Nicole Seo, Michael Rosenthal, Maksis Knutins, Zack Polizzi, James Si- mon, et al. Avalon: A benchmark for rl generalization using procedurally generated worlds. In NeurIPS, 2022.
10

---

Published as a conference paper at COLM 2025
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of artificial intelligence research, 2013.
Andrzej Bialecki, Natalia Jakubowska, Pawel Dobrowolski, Piotr Biatecki, Leszek Krupiński, Andrzej Szczap, Robert Biatecki, and Jan Gajewski. Sc2egset: Starcraft ii esport replay and game-state dataset. Scientific Dala, 2023.
Blizzard Entertainment. Starcraft II. Video game (PC), 2010.
Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, al. Do as can, not as i say: Grounding language in robotic affordances. In CORL, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurlPS, 2020.
Jianming Chen, Yawen Wang, Junjie Wang, Xiaofei Xie, Qing Wang, Fanjiang Xu, et al. Understanding individual agent importance in multi-agent system via counterfactual reasoning. arXiv preprint arXio:2412.15619 2024a.
Justin Chen, Swarnadeep Saha, and Mohit Bansal. Reconcile: Round-table conference
improves reasoning via consensus among diverse llms. ACL, 2024b.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents. arXiv preprint wwXX::2308.10844 2023.
André L. Delbecq and Andrew H. Van de Ven. A group process model for problem identifi- cation and program planning. Journal of Applied Behavioral Science, 1971.
Danny Driess, Fei Xia, Mehdi Sajjadi, Corey Lynch, Aakanksha Chowdhery, Jonathan Hoffman, Yue Hu, Sergey Levine, Vincent Vanhoucke, Quan Vuong, et al. PaLM-E: An embodied multimodal language model. ar preprint arXiv:2303.03378, 2023.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improv- ing factuality and reasoning in language models through multiagent debate. In ICML, 2023.
Linxi Fan, Alane Suhr Xie, Ziyu Jiang, Alice H To, Shuang Li Yao, Marios Skreta, Vincent Yu, Yitao Bai, Zifan Wang, Kurt Shuster, et al. MineDojo: Building open-ended embodied agents with internet-scale knowledge. In NeurlPS, 2022.
Kanishk Gandhi, Dorsa Sadigh, and Noah D. Goodman. Strategic reasoning with language models. arXiv preprint ar Xrx::22005.11 19165, 2023.
Sihao Hu, Tiansheng Huang, and Ling Liu. Pokellmon: A human-parity agent for pokemon battles with large language models. ar preprint arXiv:2402.01118, 2024.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large language model society. In NeurlPS, 2023.
Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, and Yitao Liang. Jarvis-vla: Post-training large-scale vision language models to play visual games with keyboards and mouse. arXiv preprint arXio:2503.16365, 2025a.
Zongyuan Li, Chang Lu, Xiaojie Xu, Runnan Qi, Yanan Ni, Lumin Jiang, Xiangbei Liu, Xuebo Zhang, Yongchun Fang, Kuihua Huang, and Xian Guo. Hierarchical expert prompt for large-language-modde An approach defeat elite ai in textstarcraft ii for the first time. Xiv preprint ar arXit:2502..1122 2025b.
11

---

Published as a conference paper at COLM 2025
Do Long, Duong Yen, Luu Anh Tuan, Kenji Kawaguchi, Min-Yen Kan, and Nancy Chen. Multi-expert prompting improves reliability, safety and usefulness of large language models. In EMNLP, 2024.
Weiyu Ma, Qirui Mi, Yongcheng Zeng, Xue Yan, Runji Lin, Yuqiao Wu, Jun Wang, and Haifeng Zhang. Large language models play starcraft ii: Benchmarks and chain of summarization approach. In NeurIPS, 2024.
Marvin Minssy, The Society of Mind. Simon and Schuster, New York, NY, USA, 1986. ISBN 978-0-671-65713-0.
OpenAl. Gpt-Ao mini: Advancing cost-efficient intelligence. https://opent: httpoppeaai.cmminneee/....e wtt.:co//iiiiiii com/index/ spt-4oominn-advvaccing haaaliaa-aaannnnnnntoo-eeeiiiii- www.feiiietiinnllleeeeeeeeeeeeee 2024. Accessed: 2025-03-20.
Xiang Ouyang, JeffeyJy Wu, Xu Jiang, Diana Almeida, Carolyn Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Frederick Kelton, Lilian Miller, Christopher Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurIPS, 2022.
Philip Paquette, Yuchen Lu, Seton Steven Bocco, Max Smith, Satya O-G, Jonathan K Kum- merfeld, Joelle Pineau, Satinder Singh, and Aaron Courville. No-press diplomacy: Modeling multi-agent gameplay. In NeurlPS, 2019.
Chanyoung Park, Gyu Seon Kim, Soohyun Park, Soyi Jung, and Joongheon Kim. Multi- agent reinforcement learning for cooperative air transportation services in city-wide autonomous urban air mobility. IEEE Transactions on Intelligent Vehicles, 2023.
Siyuan Qi, Shuo Chen, Yexin Li, Xiangyu Kong, Junqi Wang, Bangcheng Yang, Pring Wong, Yifan Zhong, Xiaoyuan Zhang, Zhaowei Zhang, et al. Civrealm: A learning and reasoning odyssey in civilization for decision-making agents. arXiv preprint ar arXiv:2401.1066, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 2020.
Xiao Shao, Weifu Jiang, Fei Zuo, and Mengqing Liu. Swarmbrain: Embodied agent for real- time strategy game starcraft ii via large language models. arXiv preprint ar X/Xr:22401.11799 2024.
Mohit Shridhar, Xingdi Yuan, Marc-Alexanda Cote, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interac- tive learning. In ICLR, 2021.
Vighnesh Subramaniam, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains. In ICLR, 2025.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IROS, 2012.
Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observation. In IJCAI, 2018
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Edward Lockhart, al. LLaMA: Open and efficient foundation language models. arxio preprint XIXi::3302.1397, 2023.
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. Two tales of persona in llms: A survey of role-playing and person- alization. ar Xiu preprint arXit:2406.01177, 2024.
12

---

Published as a conference paper at COLM 2025
Yusuke Urakami, Kazuya Yoshida, and Takashi Tsuji. Ilbit: Imitation learning for robot using position and torque information based on bilateral control with transformer. arviv preprint ar Xio:2401. 16653,,11.,.... 2024.
Oriol Vinyals, Igor Babuschkin, Junyoung Chung, Michaël Mathieu, Max Jaderberg, Woj- ciech Marian Czarnecki, Andrew Dudzik, Brandon Houghton, Tobias Pohlen, Valentin Dalibard, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 2019.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024.
Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. Science- world: Is your agent smarter than a 5th grader? In EMNLP, 2022.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
Philipp Wu, Yide Shentu, Qiayuan Liao, Ding Jin, Menglong Guo, Koushil Sreenath, Xingyu Lin, and Pieter Abbeel. Robocopilot: Human-in-the-loop interactive imitation learning for robot manipulation. Xrv preprint arXiv:2503.07771, 2025.
Yi Wu and Zhimin Hu. Llms are not good strategists, yet memory-enhancee agency boosts reasoning. In ICLR Workshop on Reasoning and Planning for LLMs, 2025.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report. ar preprint ar Xio::2447 106771 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. Xix preprint ar Xri:::2505.009 0.3888, 2025.
Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. arXiv preprint Xix::1911..1033, 2019.
13

---

Published as a conference paper at COLM 2025
A Brief Introduction of The Game of StarCraft II
StarCraft II (SC2), developed by Blizzard Entertainment, is real-time strategy (RTS) game renowned for its depth, complexity, and strong presence in the e-sports arena. Players choose from one of three race: Terrans (humans), Protoss (technologically advanced aliens), or Zerg (rapidly evolving lifeforms). Each offering unique units and strategies that demand varied approaches to resource management, base building, and tactical combat.
Notably, the level of micro-control (unit-level precision) often ranks Terran at the highest, followed by Zerg, and then Protoss. This trend is also reflected in our benchmark results, where Terran shows the lowest rule-based micro-management performance, Zerg ranks in the middle, and Protoss achieves the highest.
Key Gameplay Elements
Unit Coordination and Macro-Managementt Selecting appropriate build orders and unit compositions while managing expansions, production cycles, and researching key technologies is crucial for establishing a robust economy and effectively countering the opponent's strategies.
Balancing the collection and allocation of minerals and vespene gas is crucial for effective unit production and technological advancement.
Resource Prioritization:
Fog War and Scouting: Restricted visibility compels players to deploy scouts or specialized units to gather intelligence, adapt strategies, and anticipate opponent actions.
Dynamic Battlefield Adaptation: As the game progresses and the battlefield evolves, players must continuously adjust their tactics to stay ahead of emerging threats and opportunities.
B Previous Work: TextStarCraft II (Ma et al., 2024)
TextStarCraft (Ma et al., 2024) the first environment for evaluating L.M--bsed agents in real-time strategic scenarios within StarCraft 11. By leveraging Chain of Summarization (CoS) technique enhance rapid decision-making, the authors demonstrate that most tested LLMs surpass Level 5 built-in AI.
C Implementation Details and Experimental Hardware Setup
C.1 Environment and Game Settings
Game Version: We conduct all experiments using Patch 5.0.14.9333. of StarCraft II. We use the built-in Al provided in the game, and the difficulty settings are detailed in Table
Game Difficulty Level
Blizzard Difficulty
Very Easy
2
Easy
3
Medium
4
Hard
5
Harder
Very Hard
7
Élite
Cheat Vision
9
Cheat Money
10
Cheat Insane
Table 7: StarCraft II Built-in AI Difficulty Levels.
14

---

Published as a conference paper at COLM 2025
Map Selection: All evaluations performed the Ancient Cistern LE maps from the 2023 StarCraft esports 1v1 ladder.
Model Temperature: To allow for moderate level of diversity in responses, set the temperature parameter to 0.7 for both the imitation agents and the strategic planner.
C.2 Experimental Hardwares
The imitation agents are fine-tuned on four NVIDIA H100 80GB GPUs, requiring approximately six hours in total. During fine-tuning, LoRA modules added for efficient adaptation of each agent.
Training:
Inference: We a single NVIDIA A6000 40GB GPU to concurrently run three 1.5B- parameter imitation agents during evaluation.
D Details About TEXTSCII-ALL Evaluation Environment
The code is based on the python-sc2? library. Python-sc2 is an open-source Python frame- work that allows developers to create StarCraft AI bots by providing convenient APIs for controlling and observing the game environment.
D.1 Agent vs. Built-in AI Matches
We categorize the action space into three groups-unit production, building construction, and technology development (see Fig, 6 for details). The Protoss have 58 possible actions, the Zerg 61, and the Terrans 62. Race-specific commands (e.g,, Chrono Boost, Larva Inject, Mule) and few general commands (such as "attack" or "scout") operate according to rule-based logic. We implement each of these actions within our codebase, performing prerequisite checks (resource availability, required structures, etc.) before execution.
D.2 Agent Agent Matches (Agent Arena)
In addition to built-in AI matches, we conduct head-to-head encounters (" ("Agent Arena") under the same map settings and resource conditions. Because both agents operate within the same constraints and action space, these matches enable direct comparison of strategic reasoning.
E Details About Data Construction and Clustering for Imitation Learning
E.1 Details About Instruction-Tuning Dataset Construction
From SC2EGSET(Biatecki et al., 2023), identify each time step t at which action A (belonging to our defined action space) is issued. We then record the player's current state S, at time which includes supply details, unit details, building details, technology details, and ongoing commands. Notably, S, is limited to the player's information only and does not contain any enemy-related data.
At each decision point f, we pair the state S, with the action A₁ executed at time f, as well as all subsequent actions occurring within the next A minutes, forming the action sequence AL+AA- This (S,, Art+a) pair constitutes one training sample. Empirically, setting A = 3 yields the best performance (see Figure 4-()). Once this action sequence is constructed, we shift the window to the time of the next executed action L' (i.e., the first action after f), and again collect all actions that occur between L' and I' + A. By repeating this sliding-window procedure, obtain average about 200 training samples per game, A detailed overview of this data construction process is provided Figure 5.
15

---

Published as a conference paper at COLM 2025
SC2EGSet
Unit list
Probe 2210 212, 215, 216, 218, 219, 220, 221, 222, 223, Pylon: 225
"eviTypeName". "PlayerStats",
"oopp:3040,
Nexus 2209, 247]
"playerld":
"stats":
"scoreValueFooodMaad"""" 128976,
Player's current state (S,)
"scoreValueFoooUUsadt 94208,
At 2:15 game time, our currnt StarCraft II situation is as follows.
maximum capacity of We have 18 Probes. For building, we have Pylon, Nexuses and Gateway.
We have supply usage
SC2EGSet
"evtTypeNamee" "Unitinit, "loap": 3148,
"evrTypeNamme" 'UnitBom", "toap": 3152, "controlPlayerdd""
"evtTypeName"" 'UnitBom', "loop": 3331,
"eviTypeName" 'UnitBom', "loop" 6945, "controlPlayerrdd" 1,
"controiPlayerdd": 1,
"controlPlayerrd" 1,
"unitTagindex" 255,
"unffTagindex" 256, "unitTaaFFecycle.. 2,
"unffTaginde"" 257, "unilTagFRecycle" 'unifTypeName". "Probe"
"unifTagindex" 316, "uniiTagFecyccl"" 'unifT "Stalker"
"unitTagRecycde" "unitTypeNamm". "Stargate"
"unitTypeName.. 'Adept"
Action Sequence (A)++ <BBILL STARGATE> <TRAIN ADEPT> <TRAIN PROBE <TRAIN STALKER>
Figure 5: Overview of data construction process. We use SC2EGSET to obtain player's current state and corresponding action sequence. In particular, the number of units in each current state is determined by separately recording unit counts at each time step.
In order convert the action-only dataset produced by the above procedure into an instruction-tuning format that includes rationales, we use specialized system prompt alongside the GPT-4o-mini API to generate the reasons behind each decision point's action sequence. The system prompt guides the model to break the rationale into three time short-term, and long-term strategy (see Figure 7). By training on this enriched dataset, the model adopts these three perspectives to more precisely interpret the game environment, resulting in more strategic decisions.
E.2 Details About Imitation Multi-Agent Clustering
We explore multiple criteria for creating specialized agents. Among these, we generate an imitation multi-agent system based on army ratios, thereby broadening the range of possible playstyles and strategies. However, also experimented with alternative criteria, which did not yield as clear a differentiation. We include them here for completeness.
Army Composition Criterion. To encourage each imitation agent to develop a unique style, we apply k-means clustering based on the proportion of units used by each race. By focusing on each agent's unit ratio, we reveal clear distinctions among them. For instance, when clustering Protoss agents, they naturally split into three groups: one that relies heavily on Phoenixes and Colossi, another centered around Zealots and High Templars, and a third emphasizing Carriers and Void Rays.
Unit Ratio Extraction and Clustering. To capture meaningful differences in army com- positions, we focus on games that the professional player won and calculate a weighted unit distribution. Specifically, for each game, we multiply the count of each unit type by its corresponding "supply cost" (or "population value") () to account for the greater impact of high-tier units. We then sum these supply-adjuster counts and normalize each unit type by this total to obtain distribution vector of length [2/1, where / denotes the set of unit types for a given race. Formally, if n; is the count of unit type and w; is the supply cost, then the normalized ratio r; is given by:
n; X Wi
Ejeu (1; x wi)
16

---

Published as a conference paper at COLM 2025
Protoss Action Space
TRAIN UNIT:
ZEALOT,, 'ADEPT, STALKER, 'SENTRY' HIGHTEMPLARY DARKTEMPLAR VOIDRAY,
8: CARRIERR 8: TEMPEST. 10: ORACLE, E, 11: PHOENDC, 12: MOTHERSHIP,, 12: OBSERVER: 14: IMMORTAL;
WARPPRISM" COLOSSUS;, 'IIRRUPTOR;
'SILD STRUCTURE:
19: PYLON, 20. ASSMILLTORR 21: NEXUS; 22: GATEWAY, 23: CYBERNETICSCOREY 24: FORGE, TWILIGHTTOONCIL:
26: ROBOTICSFACILITY STARGATE, 28: TEMPLARARCHIVE 29. "DARKSHRINE; 30: ROBOTICSBAY,
FLEETBEACON, PHOTONCANNON. 'SHIELBBATEERY
REEEARCHH
34: WWARPGATE, RESEARCH, 35 'AIRWEAPONS LEVEL'' 36. AIRWEAPONS. LEVEL2, S7. 'AIRWEAPONS LEVELLS LEVEL2, 42: STALKER BLIN,, 43: ZEALOT CHARGE, 44: GROUNDWEAPONS LEVELT, 45: GROUNDWEAPONS LEVEL2: GROUNDWEEPONS LEVELS, 47: GROUNDARMORS. LEVELL, GROUNDARMORS, LEVEL2'
AIRARMORS LEVEL1'
'ADEPT
GLAIVES:
49: 'GRONNDRMORRS LEVEL3, 50: 'SHIEL LEVEL1: 51: 'SHIELDS LEVEL2) SHIELDS, LEVELS
53: COLOSSUS EXTENDED, THERMAL LANCE: 54: WARPPRISM GRAVITIC DRIVE: 55: OBSERVER GBAVIIICCCCCCCCCCC BOOSSTRRS RANGGEUPPRADE
TRAINN UNIT:
'DRONE, OVERLLRDD, 'QUEEN, ZERGLING; 'ROACH', MYDRALISK, MUTALISKK CORRUPTOR,
8: INFESTOR: 'SWARMHOST; 10: ULTRALISK, 11: VIPER", 12: BANELING', 13: RAVAGER: 14: LURKER.
15: BRROODORDD 16: OVERSEER*
BUILD STRUCTURE::
EXTRACTORI 22: HYDRALISKDEN; 23: LAIR, 24: HIVE, 25: EVOLUTIONNHAMBER, 26: INFESTATIONPIT, 27: SPIRE: 'LURKERDEN,
SSPAWINNPPOLL
"GREATERPIREE'
RESEARCH TECHNIQUE:
33: ZERGLING, SPEED UPGRADE, 34: ZERGLING, ATTACKSPEED UPGRADE: 35 BANEELIGG, SPEED, UPGRADE:
36 RRACH SPEED UPGRADE, 37: 'ROACH TUNNELING CLAWS: 38: 'OVERLORD SPEED UPORADE, 39: 'BURROW;
UPGRADE, 43: MELEEWEAPONS LEVEL2; 44. MELEEWEAPONS LEVELS', 45: MISILEWEAPOOS LEVELT,
MELEEWEAPONS. LEVELT,
MISSILEWEAPONS.
LEVEL1;
49: GROUNDARMORS, LEVELZ, 50: GROUNDARMORRS LEVELS' 51: INFESTOR NEURAL
NEURAL_PARASITEE PARASITE:
FLYERWEAFONS
58: PLYERWEAPONS. LEVEL2, FLYERWEAPONE LEVELT, FLYERARMORS, LEVELT,
56: FLYERARMORS LEVELZ, 57: FLYERARMORS, LEVEL3', 58: ULTRALISK ARMOR UPGRADE.
59: ULTRALISK SPEED UPORADE; 60: 'URKER, RANGE, UPGRADE: 61: 'LUREER, DIGGINO__LAND
Terran Action
'RAINN U
SSV, M'RRINEE 2: REAPER, MARAUDER" 'GHOST S: HELLION, WWIDOWINE; 'CYCLONE" SIEGETANK: LIBERATOR' BANSHEE,
'TOOR
15: BAATEEEEERR
16: COMMANDCENTER, 17: SUPPLYDEPOT: 18 REFINERY: 19: BARRACKS' 20: ENGINEERINGBAY 21: BUUKER,,
FFATTOYY: "STARPORT. ORBITALCOMMAN PLANETARYFORTRESS BARRACKSREATTOR; 27: BARRACKSTECHLAN FACTORYREACTOR; 29. "FACTORYTECHLES 30: 'STARPOHTREACTOR. STARFORTTECHLAB: GGHOSTACADEMY, 'ARMORY' FUSIONCORE; MISSILETURRET.
38 'SENSORTOWER'
TECHNIQUE:
37: STIMPACK, 30: MARINE, HEALTH
SEALTHH_UPRRDEE UPGRADE'. 39: HELLION ATTACK UPGRADE, 40: 'CYCLONE ATTACK UPGRADE",
"SMARTSERVOS. 43: SYSTEM, 46: INF WENANTTTYWEAPOSS..... LEVELY.
44: BANNSHEE SPEED UPGRADE 45: MEDIVAC, RAPID REIGNITION CEEIITIONNSSYY INFANTRYWEAPONE LEVELZ, INFANTRYWEAPONSS LEVEL3, INFANTRYARMORS LEVELL,
50: INFANTRYARMORS. LEVELZ, 51: INFANTRYARMORS LEVEL3', 52: NEOSTEEL ARMOR, 50: 'HISEC AUTO, TRACKING,
LEVEL2' 57: VEHICE EWEAPONS LEVEL3, 54: GHHSTT CL OAKING; 56 VVEHICI EWEAPONS LEVEL WSHIWWEAPONS V
SHIPWEAPONS, LEVEL1, 'SHIPWEAPONS...
61: BETTLLLUIERRREENE 62 LLIBERATOR BANEORRRRRREEEEEREEEEEEEE
Figure 6: Action space of three races in StarCraft II. Each race has actions related to unit production, building construction, and technology development. Both the imitation agent and the strategic planner form sequences of these race-specific actions to issue commands.
17

---

Published as a conference paper at COLM 2025
SYSTEM
extensive StarCraft Protoss expertise. You are given current game state professional player's chosen actions for the next minutes, Based solely these details, provide concise coaching advice three lines line each for immediate steps, short-ferm actions, and long-term stralegy in order of priority. Include specific units and their numbers where possible, and do not mention or comment on any actions currently waiting.
USER
At 04:48 game time, our current StarCraft situation is as fallows. We have a supply usage of 39 out of maximum capacily of 62. We buildings,
Nexusee, Pylons, Assimilators, Galewey. Cybemelicscore, Forge, Stargate and 4 Photoncannons. No research has been completed yet. We waiting Probe, Oracie and Stargates. <THAIN PROBE> x 3 <TRAIN ORACLE, E> <TRAIN PROBE> x <TRAIN ORACLE Es <TRAIN PROBEs <BUILD PYLON- <THAIII PROBE> X <TRAIN ORACLES <BUILD PYLON> <BUILD NEXUS> <TRAIN ORACLE> <BUILD PYLONs 2 <TRAIN PROBE> 2 <TRAIN ORAGLE> <BUILD PYLON> <BUILD PHOTONCANNON: <BUILD PYLON> <TRAIN ORACLE> <TRAIN PROBE. x <TRAIN
PROBE> <TRAIN ORACLE>
ORACLE>
<BUILD PYLON> <TRAIN PROBE>
ORACLE> <TRAIN PROBE> x <TRAIN ORACLE> x <TRAIN PROBE> x <BUILD ASSIMIATTOR: x
ASSISTANT
Immediate Steps: Focus on maximizing your aconomy by training Probes consistently; aim for 40+ Probes to ensure a streng resource
Short-term Actions: Priorilize builing additional Pylons avoid supply block and continuous production of units, particularly Oracles for harassment and scouting.
Long-term Strategy: Transition into robust amy composition by teching towards Robotics Faciltyy or additional Gateways for ground units, maintaining Oracia production for map control and harassment.
Figure 7: Prompt Details for Rational Generation in an Instruction-Tuning Dataset. We use GPT-4o-mini to generate the rationale behind the chosen action sequence for each situation. Subsequently, the generated rationale and the corresponding action sequence are combined into single output within our dataset.
Finally, we perform k-means clustering on these vectors in the unit-ratio space, grouping together games with similar army compositions. This process yields distinct army-style clusters and allows each specialized agent learn different strategic focus.
Opening Strategy Criterion. We obtain early-game build orders from publicly available metadata Among the available tags, focus those containing the word "opening" (e.g,, for Protoss, "Oracle opening," "Phoenix opening," etc.) to categorize the corresponding games, under the assumption that such tags effectively capture each game's initial strategy. However, because these tags are arbitrarily assigned by users rather than following consis- tent criteria, this approach underperforms compared to clustering by army com composition, and no clearly distinguishable characteristics emerge for each imitation agent.
Advancement Tempo Criterion. By examining when key buildings and technologies are constructed for each race (e.g., for Protoss, "Warpgatekesearch, "Fleetbeacon, etc.), we use k-means clustering to classify each game as slow, moderate, or fast in terms progression speed. Because advancement tempo is heavily influenced by resource fluctuations and in- game events, it reflects only the pace of development rather than a player's 's overall strategy. By contrast, final unit ratios capture the compositions that players ultimately commit to, thereby revealing clearer strategic distinctions.
E.3 Details About Dataset Clustering
After clustering the dataset according to the Army Composition Criterion, each imitation agent assigned a subset of the data. Specifically, for each race we obtain the following distribution (in thousands of samples) across the three agents:
Protoss: 11.5k (Agent A), 13.2k (Agent B), 15.2k (Agent C)
Zerg: 12.1k (Agent A), 17.9k (Agent B), 9.8k (Agent C)
Terran: 9.5k (Agent A), 11.0k (Agent B), 10.4k (Agent C)
Phttps://lotv. htttp:://lott..ssannnggooooo com/ wwtp:///......eee../..../aala/// ?pro only=on
18

---

Published as a conference paper at COLM 2025
F Details about Evaluation Metrics (Ma et al., 2024)
We also evaluate the performance using the APU, RUR, PBR, and TR metrics proposed in Ma et al. (2024), as shown in Tab. 8.
PBR (Production-Block Ratio): Calculates the ratio of time spent at maximum supply (200/200) to the total time, where a higher value indicates less efficient macro-management.
Measures the total resource expenditure (min- eral, gas) until the agent first reaches maximum supply, with a higher value imply- ing weaker macro-strategic use of resources.
RUR (Resource Utilization Ratio):
APU (Average Population Utilization): Averages the ratio of used population to the supply cap until maximum supply, where a higher value reflects more efficient macromanagement.
TR (Technology Rate): Calculates the proportion of completed technologies to the total available, where a higher value suggests a greater emphasis on technological advancement.
Compared to previous studies, the RUR value is relatively high, reflecting the frequent use of more expensive units, whereas the TR value is lower because only essential technologies are selectively upgraded. These slightly higher RUR and lower TR values result from an intentional strategy that prioritizes powerful units and focuses upgrades on what is truly necessary for effective combat. Notably, our method still achieves superior APU and PBR values, indicating efficient resource utilization and effective macromanagement. This combination ultimately leads to higher win rate, demonstrating that strong unit composition and targeted upgrades enhance overall performance.
However, these metrics have limitations. PBR can become misleadingly high if battles continue after the army has reached maximum supply, which doesn't necessarily mean the player is managing resources poorly. RUR penalizes compositions with expensive units, even though such choices can drive higher win rates. Finally, TR can be lower when only the most essential technologies are researched, yet upgrading everything without a clear plan can also undermine a winning strategy. Hence, relying solely on these metrics may not fully capture the breadth of LLM agents' strategic decision-making.
Method
Win rate (%) on Lv.5.
PBR (1)
RUR (4)
APU (1)
TR (1)
TextStarCraft (Ma et al., 2024) EpicStar (Wu Hu, 2025)
55
0.0781
7875
0.7608
0.4476
67
0.1211
9864
0.8123
0.2536
HIMA (Ours)
92
0.0547
15525
0.8325
0.2233
Table 8: Performance on Additional Evaluation Metrics at Harder Level (Lv.5). We present results for Win rates (%), APU, RUR, PBR, and TR against Computers on Harder Level (Lv:5). Note that lower values indicate better performance for PBR and RUR, while higher values are better for APU and TR. The reported metrics for TextStarCraft and EpicStar are taken from the EpicStar paper.
G Details about the Number of Specialized Agents
Table shows the results of evaluating HIMA at various game difficulties while varying the number of specialized agents within the architecture. As the number of agents increases, overall performance initially improves but eventually saturates, and beyond a certain point, it starts to decline. We conjecture that once critical threshold is reached, the added complexity and coordination overhead among too many agents can negatively impact overall efficiency and decision making.
19

---

Published as a conference paper at COLM 2025
Win Rate (%)
Game Difficulty
Number of agents
Lv.4
Lv.5
Lv.6
Lv.7
Lv.8
Lv.9
Lv.10
90
80
35
20
0
2
100
80
55
20
10
10
10
3
100
92
84
82
68
20
16
4
100
85
80
70
55
10
10
5
100
85
80
75
60
15
10
Table 9: Win rate by number of agents. We report the Win rates (%) of multi-agent systems with varying numbers agents across different StarCraft II built-in Al difficulty levels.
H Number of Executed Actions over Gameplay in Human Demonstration
In the early stages of the game, players face constraints imposed by limited resources and the requirement unlock specific technologies. However, the match progresses and economies and technology trees expand, players gain access to a much wider variety of tactics. This progression evident professional gameplay data (Fig. 8), which shows steady rise in the number of executed actions up to around the 12-minute mark.
Unlike existing methods (Ma et al., 2024; Wu & Hu, 2025; Li et al., 2025b) that often produce frequent, isolated actions at each timestep (leading to infeasible or redundant commands), HIMA generates structured, multi-step sequences that preserve build orders and maintain strategic coherence over longer horizons. By training professional demonstrations, HIMA captures the evolving pattern of available actions and, through a flexible rather than fixed-length mechanism, incorporates long-tern actions from the current state. This flexibility more accurately reflects the dynamic spikes action frequency throughout match, ensuring that both short-term tactics and extended strategic developments are properly modeled.
60
50
Actions
40
30
# 20
10
10
14
20
Time Interval (min.)
Figure 8: Number of actions generated per time interval. We compare the number of actions generated per time interval across 68 games from the Star League tournament held in 2021. We observe that early stages produce fewer actions due limited resources and tech prerequisites, while later stages result in a higher number of actions enabled by broader set of tactical options.
I In-depth Analysis of Multi-Agent Response Diversity
We measure the diversity of multi-agent outputs using negative log- lig-liiel metric from prior work (Subramaniam et al., 2025), where a higher value indicates greater variability among the generated actions. As shown in Figure methods that produce more diverse
20

---

Published as a conference paper at COLM 2025
responses also tend to achieve higher overall performance. This observation aligns with the "society of mind" principle, which suggests that incorporating multiple viewpoints can foster balanced mekisi in dynamic, real-time strategy settings. In particular, the "unit composition" approach eeaeaach--xliiittlll designed to promote diverse agent outputs-yields yields the greatest variety and achieves the highest performance. These findings indicate that encouraging a wider range of agent actions can lead to more robust and effective strategic outcomes.
Win Rate (%) Game Lv.7
90
Clastering
80
Opening stategy
Advenomment tempo
70
Unit composition
60
20
0.78
0.80
0.82
Neg. Log Likelihood (Avg.)
Figure 9: Agent response diversity across different clustering criteria. We measure the diversity of multi-agent outputs across opening strategy, advancement tempo, and unit composition criteria.
J Performance Impact of Rationales in the Instruction-Tuning Dataset
In Table 10, we compare the performance IL-only single-agent systems with and without rationales, where these rationales can be viewed as form of chain-of-thoughh (Wei et al., 2022). By explicitly providing intermediate reasoning steps, the model can break down complex decisions into smaller sub-decision before reaching final action. This process enhances the imitation agent's internal reasoning and yields more coherent multi-step planning, which is crucial for strategic domains like real-time strategy games. Specifically, the model learns to balance immediate, short-term, and long-term teng-termmn strategic needs, thereby enabling more robust strategies across different phases of the game.
The data in Table 10 demonstrate that including rationales leads to higher win rates across various difficulty levels (Lv.4-v.10)). This highlights the value of interpretable rationales in guiding the agent's ' decisions.
Win Rate (%) at Game Difficulty
Lv.4
Lv.5
Lv.6
Lv.7
Lv.8
Lv.9
Lv.10
Decision
80
60
10
0
Rational Decision
90
75
25
15
0
0
0
Table 10: Win rates depending on the presence of rationales. We report the win rates (%) of IL-only single-agent systems with and without rationales on our Instuction-tuning dataset.
K Details of Generating Imitation Agent's Strategic Objective
After clustering the data based on unit ratios, use the clear differences among those clusters to define a strategic objective for each one according to its unique unit ratio pattern. We use the chatgpt-ol model to examine each cluster's 's unit ratio pattern, guiding us to infer the corresponding strategic objectives. Regardless of race, when dividing the data into three clusters, each cluster is categorized respectively ground support unit focus, air unit
21

---

Published as a conference paper at COLM 2025
focus, hybrid of ground and air units. The prompt for generating strategic objectives and the strategic objectives for each race are shown in the Figure 10.
System Promet
Proteses' a Strvingi Ohjective
Here are the néto detributons for 15 Procose units a spectti compastion for
Ariinor A:
AdVieor A, B, and C.
A mix and ground unis, with cleir focus domnance (Cantiers
Phoenx Tempests).
HIGHTEMPLAA DAARKTEMPLARC VOIDRAY. CARREER TEMPEBT, 'ORACLE, PHOENIX, IMMORTAL" COLOSSUS: DISRUPTOR), ARCHON]
AOEPPT
Sentries, Archors. Refecets a late game hyord approach, blending power with limitd ground utiity unis.
Mirimal
support
Advisor A.
[0.08, 0.15, 0.01,0.01, Adiiler B:
0.01, 0,00, 0.00, 0.02, 0.27, 0.05, 0.01, 0.27 0,09, 0.04, 0.03)
Aditer Bt:
Ground-Twery composition with strong frontine units lik Zuzi ots, Stalkers, and High
6,10,
0.02, 0.20. 0,10, 0.00, 0.02, 0.05)
C:
Includes
support
Warp
[0.00, 0.00, 0.21, 0.01, 0.00, 0.00, 0.40, 0.31, 0.02, 0.88, 0.01, 0.00, 0.00, 0.03, 0.03]
Archons fixibilty
-A balanced and adaphable midegarne stretegy that allows for troms tramcitioe baced on the siueton.
Based on these natiow, please describe nach advisor's stralegic dbjectivos or overal game plan in concise point tormat, with exacty three builet points per advisor.
Each point should be singl line and offer loiiel infrrmale for these
Advikar C:
units
Ovenehe focus grounn presence.
partourarty and Cantorr, with minimal
advisor's explanation exactty three Facus explaiing how the composilion emphosozss aws cartain unit types, how thyy synergize, and what stage of the game they excct In.
bullet).
Almost investment support utllit, amph asizing raw power. Represents lote game air superirity strategyy designed to dominate the chiee with hight DPS.
Zerg's l Stuitnglo Objjection
Terren's a Strilege Objective
Advisor A:
Advisor A.
and with clear locus dommance (Musakako,
mix air and ground unis,, with clear focus on air domn nance (Botttoonulorrs Libeaator, Vikinga).
investment support or intestors Vipers.
Minimal Investment support unis Ravens Ghosts, late game hyond approsch, slending power with ground utiity unts.
Ruflects late-game hybrid approach, blanding power with linited ground and usility units.
Advisor B:
Advisor B:
Grmund--eety
trontine unte Zergings, Hosches, and
Ground composttin with strong trontine units lie Marinos, Marauders, and Stege Tarkes
Hyd-alisks.
versadle harassment support units Banellngs, Mutalisks, Nydus
Hollions, balanced and adaptable mis-game stategy allews tion sitution.
harassment support units like
Worms, or intvstors.
A balanced and adaptable mio game stratogy that allows tran mansilions into advanced wr units or Un lite.
Advisor C:
Adiitor C:
Oveneheiming foous on units, partourarly Battecnusers and Liberators, with mitinal
partcuriarty Brood
minimal ground presence.
Almost invostment support utillt, emphoslting
Almust inveetment in support unit utilly. amphasizing new offereive power. Represents late game air suporiority stratiggy, designed to dominato the skic with
Represents game superioity strategy, designed stdee high DPS.
high DPS.
Figure 10: Prompt for generating strategic objectives and strategic objective prompt for each race. We identify the underlying strategic objective of each imitation agent based on its unit ratio.
L Various Open-source Models for Imitation Learning Agent
We conduct additional experiments with various small agent models beyond Qwen-2 1.5B, and the results are summarized in Table 11. These results show that all tested models achieve similar performance in our setting, even though their performance differences may be more noticeable on other benchmarks (Yang et al., 2024; 2025; Touvron et al., 2023). We believe this is because all models are trained on the same imitation learning data we collected, which minimizes performance gaps between them.
Model
IL Agent Size
Win-rate (%) at Lv.7
Qwen-2
1.5B x 3
82 (41 /50)
Qwen-3
1.7B x 3
85 (17 (17/20)
Llama-3.2
3B x 3
85 (17/20)
Table 11: Win-rate comparison at difficulty Lv.7 for different imitation agent models.
22

---

Published as a conference paper at COLM 2025
M Various Open-source and Closed-source Models for Strategic Planner
To investigate the generalizability of SP' across various models, we evaluate multiple open- source and closed-source models, as shown in Table 12. The closed-source GPT-4o model achieves the highest performance. Among the open-source models, larger variants (egg,, Qwen-2.5 72B) perform well, closely approaching the performance of certain closed-source models (egg, GPT-4o-mini and Claude).
Model
Size
Type
Win-rate (%) at Lv.7
Qwen-2.5
32B
60 (12/20)
Qwen-2.5
72B
75 (15/20)
Qwen-3
8B
Open-source
15 (3/20)
Qwen-3
32B
55 (11/20)
Llama-3.3
70B
65 (13/20)
GP'T-4o-mini
82 (41/50)
GPT-4o
90 (18/20)
Closed-source
Claude-sonnet
70 (14/20)
Claude-haiku
60 (12/20)
Table 12: Win-rate comparison at difficulty Lv.7 across various models.
23

---

Published as a conference paper at COLM 2025
N Prompt Details
We present the prompts (shown in Fig. 11, 12, and 13) that illustrate how our HIMA agent can generate reasonable strategies and decisions within the StarCraft II environment.
System Prompt
statoglo maker StarCroft Il game, andlyzing Inputs from three diferentt Protoss advisors
stancos:
Advisor A:
A mix wi and ground units, with a diar focua on air dommnnnce (Cam bamaars Phoenox Tempeota).
mini investment in support trch unis lik Sentrics, Archons, and High Tomplans. wiih lnvited and utiiity unils.
Raalects a late hytrid approach, blending
Ground-hewry compostioon with strong fromllie Zestots, Stakkers, lighh Tempars. includes varsedie harasament and support unia Onacles, Wasp A beir aedd and adegoatle mid-game strotgy that ove transtions based the atustion.
for Bly.
Adweso Ct
Ovanwhiining foous on air units, panticutarly Vo driaas and Carriers, with mininnal ground presence.
Almost investment support to utility weiting new ottensbee posser.
cominate skies DPS.
Your is anatyze advior recommendiaions align them
straleglo options cons dering ourrent resouroes and game state.
important Notes for Decis on Making:
Commanda that are in production have already been unscuted, so never mention on them.
Any previousty dty tailed commands from the lest action sequence MUST be addtressed in your new action sequence.
Your rosporse must include all yous outlined bellw
Currett Assessment
Stage Ecanomy: Endyyhool..n game
Miltar& Teot:
Cummen forter, techenlogies, and producion cspabiliio tine
1.9 Enemy Anatysis. Known information and strateglo Imp impilcations
2. Advioor Strategy Resoluton:
2.1 Gernnning Agresd Vinepaiita
Generating Contlicted
Cantlicts
Generating isclated Viewpoints
2.5 Ganenuting g Final Strotgic Direcion
3. Bhattgy Formulation:
3.1 Core Stawegy Wain stralegiic direction based on advieor's i advriee
5.2 Unit Composition Statogy: Avaid over commimment to single unil typ
3.3 Reacurce Alsccrion:: Pr distrbution of
Senorate an sequence actions that encompasses both
planned actions
Consider actions pred cluble time-ame ( nost minutes gumapaay)
Sequence should Induce:
'mmmedilta actiora
including prerecuisites and execution of any prevously tailed actions it they extet
IMPORTANT: LOO'T DDONT MENTION ANY COMMANDS THAT ARE IN PRODUCTION
Shoo-ttem
Long-torm uctiona
Analysi Roquirements:
Before preserting the Actions Summary:
Primary Advisor [MB/C]
Ratleraw: (Eoplan why you chose thi acvisce)
Justly how your action seguence supports the choson straloglc path
Explain how you've withwooed any prevically labled commancee
Every action
from provided
TRAIN PROBE: TRAIN ZEALOT, ADEPT, TTAIN
SENTRY, HIGHTEMPLAR, DARKTEMPLAR VOIDRAY,
TTRINN CARRERY TRAIN TEMPEST, TRAIN DRACLE: LE: TRAIN PHOENIX: TTRAN MOTHERSHIP. TRAIN OBSERVER: TRAIN IANGHTAL TRAIN WARIPHHISM
TRAIN CCLOSEUB; TRAIN DISRUPTOR; RR, MORPH ARCHON, BUILD PYLON, BUILD ASSIMILATOR; BUILD NEXUS: BUILD GATEWAY
'BULD CYRERNETICCCOIE BUULD FORGE, BUILD TWILIHTOOUUGGL BUILD ROBOTICSTACLL ACLIITY 'BUADD STARGATE: DUILD TEMPL VIRAAIIIIVEEE
BUILD DARKSHRINE, BUILD ROBOTICSBAY BUILD RLEETBEACON, BUILD PHOTONCANNOY BUILD SHELDBATTERY: RESEARCH WARPOATE.
NESEAAGGH AIRMEAPONS LEVELT, HEGHEARCH AIRMEAPONS LEVELZ WESEARCH AIRINEAPONE LEVELT, NESEARCH ARARRORS LEVELL.
RESEARCH AIRARMORS, LEVELZ'
AIRARMORS_LE SEOMOORS_LEEEERR RESEARCH
RESONATINGG SLAVES, RREEEARCH STALKER
ZEALOT CHARGGE: TRESEARCH GROUNDWEAPOOS LEVELL, TESEATTHH GROUNDIWEAPONS
TIESEARCH GROURDWEEAFONS LEVEL3,
GROUNDARMORS RESEARCH SHIELDS, LEVELZ; RESEARCH SHIELDS, LEVELS, REBEARCH COLOSSUS, EXTENDED, THERMAL LANCE,
RESEARCH GROUNDARMORS. LEVELF, REBEARCH GROUNDARNORS LEVELE,
RESEARCH SHELDS, LEVEL""
RESEARCH WARFFRISM GRAVITIC DRIVE, RESEARCH OBSERVER GRAVTIC, BOOSTERS: RESEARCH HIGHTEMPLAL PSISTORM.
RESEARCH VOIDRAY SPEED, UPORADE RESEARCH PHOENIX, LPGRADD, RESEARCH TEMPEST GROUNDATTACK UUPORAE
Each action must enclssed in angle bradkets Mudtple actone forme <EXACT, ACTION N
Begin your
Summany" Iellowed by your recommended sequence
Figure 11: System prompt for the Protoss race. It consists of two parts: one that guides each imitation agent's strategic objective, and another that supports the reasoning process of the strategic planner. For other races, the system prompt remains unchanged, except for the list of available actions.
24

---

Published as a conference paper at COLM 2025
Intstion Agent's Input
6:6:22 game time, our cameet StarCraft i tuation is tollmss We have a hpppy usage of 87 out a masimum eapacity of 108. We have 48 Probos, 2 Zealots, Adopss, Stalker and Voltray.
For buildins,, have Neouses, Pylore, Assimlatore, Galawayy, Stargaiie and Cybemelisccre. have researched Worpgate.
Probes, Zealet. Acept, VVodday.
Strategss Planner's Insut
6:32
StanCrafi is suatton follows. have
of capucity
Stalkor Gatteways, Stargate Cysemettcccee. d6 Probes, Zeakits Pylone, have rosearched Worpgate, watting Probes, Zealot, Acest, deid
The advisors provided dfferent suggestions:
adh sor A sugpested
Immediate Steps: Focus stablizinng your economy training more Probes tup 50) to ensure strong resource Income for future unit production.
Shart-Temn Term Actions: Torralioo into carieehheayy compostton cortinucualy prodicing Viiddaps and Caniers while building additional Pylani avaid supply block. and Sentried counter threats expansting economy Necuses
TTrm Strategy: Aim
force with 5-7 Carders supported
Assint
my addice <TRAIN
PYLON- -TRAIN cRUL D ASSIMI ATORt, <TRAN VOIDRAYS <TRAIN PRODES 2 <TRAIN VOIDRAYS <TRAIN
VOIDRAYS -BULD ASSIMILATOR- -TRAIN PROBE- VOIDRAY- -BULDD
PYLONS <RESEARCH AIRWI APONS. LEVEL'S> <TRAIN PRODES
PROBES -BULD PYLONS <TRAIN PROBE- x4 KTRAIN ADEPT> <TRAIN VOIDRAYS <TRAIN PROBES -BUILD ROBOT -TRAIN VOIDRAY. <TRAIN PROBES <HULD NEXUS: -BUILD PYLONS 2 JIRAIN - =BBUILD
PROBES 2 -BUILD FLEETBEACONN <TTTAIN PRIOEES GUULDSTAFGAA
SHELDBATTERY, <TRAI PROBE: x -BUILD PYLON- <TRAIN CARRIER <TRAIN PROBE- -BULD ASEIMLLAOR- <TRAIN ZEALOT> -BULD ASEIMLLTOR
<TRAIN PROBE>
sugpeeled
Immediate Steps:
saturaing your economy by training queued reach optimal and gas accumulation ensurtng you can support larger amy. Short- Actions: Prin line suilding adettionl Pylens to avai supply block and maintain unit production, while also deploying an Orthotomial mup control hanaasment Long-Term Strategy: Transtion on into strong air force by continously producing Carterrs from your Fleet Beason, while upgrading your weapons to enhance your unit
uffechhonusa for mid-gan mid-game engagements. to.
adicc is <TRAIN PROBE- x 3 ADEPT> x REEEARRHH VOIDRAY 3PEED, UPGRADE> -UILD PYLONS <TRAIN PROBE: <BULD SHELDBATTERY: <TTRINN PROBE> <DULD PVL ONS <TRIINN CARRER <TRAIN PRODES dIULD PM <BULD NEXUS> <BULDD <TRAIN CANR Ch <DUILD ASEIMILATOR= <TRAIN CARRIERS -BUILD PHOTONCANNON: -BUILD SHIELBAATTEYY-
PROBE- <TRAI CARRIER. STARGATE- PHOBES
PYLONS <RESEARCH PROBE. -UILDD PHOTONGANNON- -TRAIN PROBE. <BUILD PYLON- <TRAIN PROBE+ x -BULDD ROBCTICEFACLITT- -TRAIN PROBEX -BULD PHOTONCANNONO <TTAIN MOBEE <BULD PYLON-
and advisor C suggasted
Immediate Steps: Focus on salureting your mineral lines by training al awallable Probes to maximize your esonom;; aim for 50: mineral acc.mulation Sharl-lrm Actiona: prepanttion for petential push, ansure you have hew of unita by warping addtiionl Zealsto and Sarkers in whso planning fora Rapid fire
Strategy:
balenced compesiton
by in
while expanding
your economy long-term.
So my advic in <THAIN PHOIEE <THAIN VOIDRAY> <RUILD GATEWAYS <THAIN PROBES 2 JUILD GA GATEWAYS <THAIN PHOBES 2 -BULD ASSIMIATOH
<TRAIN PROBE- x <TRAN ZEALOT> 2 ARRAIN -TRAIN VOIDRAYS <TRAIN OESERVER: -BULD NEXUS <TRAIN PROBES x -BULD PYLON- <TRAN PROBES .TRAIN VOIDRAYS -TTIIN OESERVERSS <BULD PYLON= X2 -TTAIN PROBE: <BUILD PYLON- 2TRAIN PROBES x <BULLD
PRBERNETICSCORR -TRIIN --BULLDYYLOO PYLON- <TRAN PROBE- .REEAARCH GROUNDWEEPONSS_LE LEVELTo eTRAIN VOIDRAYS <TRAIN PROBE> <BUILD PYLOND eTHAIN PROBES CIRAIN VOIDRAY <BUILD PYLONO <IRAIN PROBEE
PROBE- X2 -UILDD PYLON <TRAIN PROBE. <TRAIN STALKER <TRAIN PROBE- X2 -BULLD GATEWAYS 2 -
following
YOUR PREVIOUS RESPONSE CARRIER, FAIEED BECAUSE FLEETBEACON NOT
Figure 12: Input example for the Protoss race. The imitation agent's input consists solely of the current game state. The strategic planner's input includes not only the current game state but also the imitation agent's output and feedback from the environment.
25

---

Published as a conference paper at COLM 2025
Iitstinn Agent's Durpit
Immediate Stepe: Focus on stati tring your economyy tririing more Probes up 60) to aesure stoong resoure tromm for future unt production
Short-Tern Actions: Transition into cartior hoavy composition condrucusty prosucin Veldrays and Canters whil also building odutiinal Pylons avaid suppty bleck. Sentr counter treess espanding
ong-TTrm Strategy: Amm rebuat lorce with 5-7 Carrers susporied
Nicc.oee
Assimiltoors.
my -RESEARCH AIRWEAPONS, LEVELTS
PYLON-
PROSE> 2 <TRAIN VIDDRAYS- PROBE PROBE> -BULL ROBOT <TRAIN VOIDRAY- <TRAIN
<BULD ASSIMILATOR-
PROBES =BULD
<TRAIN VOIDRAYS
PROBE. -BULD PYLON> XTRAIN PROBE> KTRAIN ADEPT> KTRIIN VOIDRAY TRAIN
PROBES x -BUILD RLEETBEACONS- TTRIIN PROBE: <BULD STARGATE: STRAIN PROBES <BULD NEXUS> -BUILD PYLON- 2 <TRAIN PROBES -BUILD
SHELDSATTERY. <TRAIN PRODE+ X +BUILD PYLON XTRAIN CARRIER X <TRAIN PROBE. <BULD ASSIMILATOR- <TRAIN ZEALOT> -BULD
ASSIMILATOR- <TRAIN PROBE>
Strtegle Cutour
.BUILD PYLON>
<TRAIN PROBE-
<TRAIN VOIDRAYs
-BUILD ASS/MLATORR
<THAIN PRODES
<TRAIN VOIDRAY-
AIRWENOONS_
<TRAIN PROBE>
-BUILD AESIMLATOR.
<TRAIN VOIDRAY:
<TRAIN PROBE- 2
<TRAIN VOIDRAYS
<TRAIN PROBE>
<BUILD
-TRAIN VOIDRAY:
<TRIIN PROBE-
<BUILD ROBOTICSFACLITY
<TRAIN VOIDRAY-
<TRAIN PROBES
FLEETGENCON
PROBES
-BUILD NEXUS>
-BUILD PYLONS x2
<TRAIN PROBES x
-BJIDD SHELDBATTERY:
<TRAIN PHOBE>
-BUILD PYLONs
-BUILD ASSIMLATORS
<TRIIN ZEALOT>
-BUILD ASSIMLATTOR
<TRAIN PROBE-
Figure 13: Output example for the Protoss race. We demonstrate that the strategic planner's output currently includes only its decisions, but by modifying the system prompt, can incorporate the rationale into the output as well.
26

---

Published as a conference paper at COLM 2025
O Qualitative Results of the Feedback System
We illustrate how the HIMA architecture responds in two representative feedback scenarios. Figure 14 shows an infeasible action being received, while Figure 15 demonstrates coming under attack.
At 8-2 game time, our qureet StarCrett it ottustion is as folloss We have supply usage of 138 out a madmum sapacity of 142. We have 50 Probes, 3 Zesicte, Adepte, Stalker 10 Vordrays and Canter
For buklings, ase have 14 Pyfons 4 Assimiaion, Nesux, Gannaay. Cyytemeltscans, Forge. Rabertcehedily Steegstes and Flleebbazon. Wie hevn nowartted A nesapore isoel, 1. We are waiting 1 Neeeu. NEXUS will be completed in sacands
Agunt
Agent B
Agant
Inmediate compleiing
Sleps: Pylnn yuppy
Immediate Stepe: Focue conursting your economy by ng
optimal
Shurt-Tarn production
Actiona:
gethering
your any
Short-emm sgganel aegrese
lote) theraikgning sbengthen
provdee support.
Long- Stratpy
strone
continung produce
Strategy sggrsshely
nupport.
Etergaan
your
cang-temm Stratagy:
strong armmy
compasilion Bay your
upprades
meddining
upgrading your
cBUILD PYLONo <BUILD STARGATES
seapprt
your
ASSINIATOO
prep mak-game push against gro
PYLONS <TRAIN GARREED-
PROBE -BULD STARGATE>
<BULD FORGES CARRIERS
<RESEARCH AIRIEEAPNN LEVELZ> VOIDRAYS cTRAIN PROBES
PROBES
<RESEARCH AIRVEAPONS
ATORl CANNEETS
Feedback Your rescorae cl FUUN CAVIIEA was falted becuuse Pyton eest.
Agprepalan)
Strategy
Strategy
Shage
Gamenning Agreed amphasiie
Storegy:
Stage: Early/ia game. oning ateedivyy
an composiioon
present supply
through
rg selurelann expecially upcamiing cxmpletion
aupply
supports production.
Pykes..
suggests
Stratagy:
denenaing Conticted Vicwpairts
Support: support.
Gurnent
suppoota Adkiect
Voktrass,
Allocaiont
1
Probes
Gobemates Core,
stutagy.
that susply Lisoks.
Bargelas, Fiuil
Wisspons
Russo Conflicta
building additionl
Enemy Analysis
Ganeways dominancs smatagy suggessed Adkisors
composition for ground nued balanced
Dalinns from
engagements.
Genoraing Viowsokktss
Phiorftzes production and dominanne.
Foouses units.
fower
Advisor Advocates strong economy
defensive
Senercting Final Staatgii Dircct
Direction: Following Advlsor
foous
Velldays
Carviers while ensuring build the necessory production
infrsinusture support
Planning:
The TRAIN CARNIER due availabre We Pylon improve affectveness.
Actors:
Tem Continue acdifinal
Rinal Artions Sunmary:
Piimaay Advibor, AdMssr A
Rational: Addder As ovanogy aline bost with our oumnnt state, emphosicing air dominance and the necessary infrestructun une to support oustained unit production. Additionllyy noopporates the need for producion effic thro igh Gareways and addrosscs the supply issue
Justiicatoon The sequence of actions alms to ressive provious failed commandds and ensures a strong air fo while maintaining the abiiity produce units etfctiveey.
Recommended Actions:
x 2 BUILD FORGE- XTRIIN VOIORAY+ ATRAIN PROBE. X VOANN DO ZEALOT- x VOIERAY. .BULD GATEWAY. <TRAIN PROBE. XTEAIN VOIDRAY .UULD PYLON- <TRAIN VOIDRAY & <BULDD PYLON- TRAIN PROBE. 2 -BULD SHELDBATTERY.
.BULD OLLON- <BULD NEXUS> TTRAIN CARR ER
.TRAIN VOIDRAY> 2 BUILD PHOTONGAMRON x2 TTRAIN VOIDRAY TTAIN OBSERVER TTRAIN PROBE> 22 TTRAIN VOIDRAY.
TRAIN PROBEX TTRAIN 2
Figure 14: Overall Strategy Generation Pipeline in an Infeasible Action Scenario. Strategy planner (SP) arranges the action sequence so that the prerequisites for an previously infeasi- ble action are fulfilled, allowing the action to become feasible and eventually be executed.
27

---

Published as a conference paper at COLM 2025
game StarCrell
suppy Wi have 76 Probes, /.zeeto, Adepts, Sto to vedrays
eapeckty 150.
For buildings, we have 15 Pytons, & Ansimittions, Netuses, 2 Gateways, 1 Cyteemeticscore, 1 Farge Twilghtooundl, 2 Stargates, 2 Photontannons and 5 Sheldtatters.
We have researched Warpgate. We are waring for nothing in...
Agent
Agent
Agemt
Inmediato Stops: Fecus madniding your economyy by haining Probeso comsistently aining for around Frobes to your beses.
Inmodiile Stops Build additional Pyleno susply and ensure continuuus unt productoor pria lize
Inmedditt Steps: Facus un buising witiiimaa Pyere avcid supply bock as you see asproacting your macmum
Pylero inmetiatlly.
capecity
Thrr Actions: iee your Voldrass soout and harass; considar adding more Adpps Zoolets support your anyy compusilion agains ground unit parttaiing your dominance Long Term Strotegy: Transit on balanced army composatio with a
Shorth Term Actions: Focus traniigg more Vordras (ain 15 and hannttooiigg Disngtorss syttegg your
Short: Actions: Trann more daya and Stakers to
bobeas your groand army, essuiig you can courner poletrial Irreato
onsure you have onough Prob
raan map corteo.
salurite your cconemyy.
Long- Strategy Trarstion shong
Long Strotegy: Extablish strong air force
comsositon with Carriers supsort from Adspts
Vodiays, ets and Acopls:
Vilenye and Disrutters while oxpanding your oconomy
Zea ots expanding your aconomy more Nocuses and Probus mainaan apong soonomy for d-game usgraciss
upgrada your units Air Weapons Zoalet Charge while buikting ad01 Gateways strongor ground prosance
with anditional Nosuses; considor socning into Rocolico compodtion
cornol..
my advice TRAIN
x So adiice -BULDD VOIDRAY-
So my adeice -TRAIN VOIDRAY. <TRIN PROBE. x2 -BULDD PVLONw -TRAIN
REBEARCH AINNEAAOON LEVELI>
-TRANN PROSES
VOIDRAY,
.TRIN PROBE. 4 -BUILD ASSIMILATOO-
Feodbaokk ATTACK!
Rouch,, Hydhuisk
Appregation
Assesament:
Advisor Stratogy Resolution:
Stratogy Formulation:
Shage Sconony
2.1 Genunainng Agroad Viewpairts.
Core Shanogy:
Brage Miligams We cunurtly on olfenaive
A erigh estors balancing our anmy with more
Focus masemiing ig our foros Voktnago while imagguting ground (Zealots Adepts) deal with ony's ground foross effectivey.
against the enemy Zug forces.
Adupss Zeaters ground support ort while maintaning air dominance.
Recources: cconemy solld 74 Probes, allwwing brrookinuuus untt production and upgrades.
Aovioer suggests fecusing on building Valahays nor satioon ting Dissuporrs which symorgite with our
Uni Compos fon Stratogy
Millar Tect
all un'ts
Condinue to produne Vordaayo to
skios
Curentt Forces.
Advisor highights building more Voldras and
and with
Hydralivva
Zealtt Adops,, Stalkers, 10 Vold
kors, rearlordng and ground counter threats.
Gecund: Zealets and Accpts absorts dumage
engage onemy units in clase ocmbar
Pylons (ampls supply capacit).
2.2 Se Senorading Conficttdd Vioppiiss A Adopps for ground prosencee, swards Disrupooss sttack stritgy.
Support: Shrots Battriiss for Photon
sugpost building Zealets
dellons base.
Nouusss, Gateways.1 Cybeeetic Coro, Farge figh Councll Stargates Photon
may not 1t sument
Rescurce Allooadoo:
Canmons, and Shiels Battries
Resouroes will directed towards training more Velorago and ground white malimaiiing our economy through Probes.
Upgrades: Warrgate rescarch completed, but no
Aaviser A suggests maximiting ng cconomy through te tocusing no on innidiate litary onto
splcoto weapon upgrndss montioned
conflciing g in mid game atack
Enemy Analysik:
Enemy Composttor: Readhes Hydraliske, and Ravagere.
2.3 Reso ving Conficts:
Seven immeciate attack siuuddon, we prientize actiont
impiicattne:: enemy compoottion primaity ground some sapatiitiss les (+tydoilskk)) Vordaae
current haary enemy's
includes ing more Voltnye to
againet Hythral need coutiove their fir and potenttal Comosive Ravagere.
Reaches Hydhaallkk
ring ground
support
can notrmes economy conserne, fecur
on ourren engagemment
optimiting amy
Cerwening Vinappontte: Smphouiize Dienuptors synerg
ground suppoet
Supports strong -io-ggddd
Gemeeniin Stovlgge
printiee immedate tary
4. Action Planning:
immediaa Actios: Given wa are on szack, the first priorit to train andil Volcnegs and ground unita to beister our curont army composiiion
Short Torn Acdorss: Continue to zain Probus as necced but foous pr marky on mi tary units for the atadi Use the Voleaas to soout and haraso the enamyy while ongaging directl in the f2tt.
Long Tom Actins, Aftor the ongagement, co sider upg sading our units and cxponding our coonomy futherr with more Nouses and Ascini uooro.
Final Actions Bunmary:
Prinayy Ackisor: Adviso A
Rationale: Addtor A aligns woll with our ourren strttgy of maintaning air dominance while acding ground units to support our attokk against the onemy.. Justitcation: The action soquenos is dosigred to mosim zo our millia itayy prosonoo for the ongoing assoult, anduring we mar mannain pressure on the oncmy while considering future coonomic stability.
Recommended Actions:
TRAIN VOIDeAY. 4 TRAIN ZEALOT x BUILD PYLON <TRAIN PROBEX X TTRINN VOIDAAY x -BULD PHOTONSANOON- TRAIN PROBE. <TRAIN PYLON-
-BULD PHOTONC-WNON- BUILD SHELDOMTTERY TRAIN VOIDRAYs -UILD SHELDEATTERY <TRIN PROBE xa XTRANN VOIDRAY AB LUULL LASIR.
TRAIN PROBE- -BULD MLLATO. TRAIN PROBE x2 .BULD STARGATE TRAIN PROSE xe BUILD FLEETBEACON TRAIIN PRODE x -BULD PYLON, .PANN VOIDRAY. 3 BUILD CYBERNETCGGOREE TRAIN VOIDRAY .TRAIN ZEAL 2 ATRNN x2 -BULD PYLON. PROBE- x -BULD PYLON>
Figure 15: Overall Strategy Generation Pipeline in a Defense Scenario. Strategy planner (SP) urgently produces counter units (Void Rays, Zealots) in response to the enemy's attack, and also constructs defensive buildings (Photon Cannons) to prepare for subsequent assaults.
28


Reviews:
[{"review_id": "hzkDDBzkxd", "reviewer": "Reviewer_bnMe", "summary": "This paper proposes a hierarchical multi-agent framework (HIMA) for strategic reasoning in real-time strategy games, specifically focusing on StarCraft II. The framework utilizes specialized agents and a meta-controller to achieve state-of-the-art performance compared to LLM-based approaches. Additionally, the paper introduces the TEXTSCII-ALL environment for evaluating performance across all race matchups in StarCraft II.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "**Empiricism, Data, and Evaluation**: The paper demonstrates that HIMA achieves superior scores compared to other LLM-based approaches, providing a strong empirical foundation. The introduction of the TEXTSCII-ALL benchmark is a significant contribution, as it enhances the evaluation of performance across various race matchups. Furthermore, the incorporation of tactical rationale into existing datasets using LLMs is a novel contribution that enhances the instruct-tuned dataset with medium- to long-term goals.\n\n**Technological Impact and Understanding Depth**: By allowing agents to output action sequences, the framework promotes more consistent behavior, which is crucial for effective decision-making in complex environments. The use of clustering to analyze data and imbue agents with specialization is a noteworthy aspect of the work, and this automation could be applicable to other problems, enhancing its relevance. This ambitious approach to tackling complex long-term decision-making with LLMs highlights the potential for significant advancements in the field.\n\n**Efficiency**: The reduction in the number of LLM calls due to long-term planning makes this approach applicable to real-time scenarios, which is a significant advantage.", "reasons_to_reject": "While the paper presents several strengths, it is limited to a single benchmark (StarCraft II), raising questions about the generalizability of the findings. Future work should explore the application of HIMA in other real-time strategy games to validate its effectiveness across different contexts.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PvU19IfaX9", "reviewer": "Reviewer_RWbE", "summary": "The paper presents a new method for the text version of StarCraft II with LLMs. The authors introduce a hierarchical agent with a meta-controller (strategic planner) that produces mult-step actions. These actions are first proposed by multiple agents with different playing styles and then aggregated by the strategic planner. The authors also introduce SCII-All, which extends SC2 evaluations to all race conditions.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Compared to previous baselines, the method seems to work well.\n- The ablations are clear and make the role of certain parts of the method. I like the analyses of temporal cot, planning horizon vs win rates with api calls.\n- Comprehensive benchmark expansion to nine race combinations.", "reasons_to_reject": "- The paper is very unclear. A lot of the sections of the paper, in particular, the method, are difficult to understand. For example, terms like Nominal group technique, Strategic Objective aren't explained, and datasets central to the method like SC2EGSet aren’t explained well.\n- The problem is not well motivated and the design decisions seem arbitrary at points.\n- No error bars or significance tests.\n- The method uses GPT-4o, a closed-source model. No open-source/open weights model is tested.\n- In Table 4, with the strategic planner being GPT-4o, the configurations aren’t compute matched. This table felt a little misleading initially, as the model size column doesn’t include the size of the strategic planner.\n- Only one model has been tested as the strategic planner? Does the method transfer to other models?\n- The method seems very specific to improving performance on StarCraft. There is no discussion on why the techniques introduced are general or why they would work for domains beyond StarCraft.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TKrxbgrYms", "reviewer": "Reviewer_crGH", "summary": "This paper proposes a hierarchical multi-agent framework to enable more general and adaptable embodied intelligence in robotic systems. The authors integrate multiple specialized sub-agents that operate in parallel and are coordinated. By imitation learning (actually is SFT) different LLM agent with different roles, they demonstrate the feasibility of this approach using a virtual embodied agent.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1), The topic is interesting. Basically, using LLM agets for emboded AI is interesting research area. The work is focusing on StarCraft II.\n\n2). The work builds the pipeline with imitated LLM agents (SFT) different roles is proved an effective way.\n\n3). Experiments validate the effectiveness of the proposed approach.", "reasons_to_reject": "1.It is unclear how well this architecture would scale outside the simulation environment or handle more complex physical tasks.\n\n2. The paper is not well-formulated and very hard to follow. Many terms are used without well definition.\n\n3. The idea basically is not very novel. Actually, the idea is not exact imitation, actually is role-wise fine-tuning (SFT), which is not new in the literature.\n\n4. The work only use one base model, i.e.,  Qwen-2 1.5B, in the evaluation. It is suggested to include more base models.", "questions_to_authors": "The work only use one base model, i.e.,  Qwen-2 1.5B, in the evaluation. It is suggested to include more base models.", "extra_scores": {}}, {"review_id": "QvQ9tv8dlh", "reviewer": "Reviewer_W96F", "summary": "This paper introduces a hierarchical imitation multi-agent (HIMA) framework for StarCraft II (SC2), where specialized agents create structured action sequences, and a meta-controller integrates them into cohesive strategies. By leveraging human replay data with tactical rationales, their approach maintains build orders, minimizes LLM queries, and adapts effectively to changing conditions. Additionally, this paper also presents the TEXTSCII-ALL environment, encompassing all nine race matchups, as a comprehensive testbed for real-time strategy (RTS) games. Experimental results demonstrate that HIMA outperforms state-of-the-art (SoTA) baselines in both win rates and efficiency, highlighting the effectiveness of combining imitation-driven specialization with high-level coordination in complex real-time strategy scenarios.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper introduces a efficient framework (i.e., the HIMA framework ) that deploys specialized imitation agents trained on human demonstration data for generating diverse structured action sequences, coordinated by a Strategic Planner (SP), to address the limitations of existing LLM-based methods, such as producing short-horizon actions and invalid build orders.\n\n2. This paper implements experiments to demonstrate that the proposed framework surpasses built-in AI opponents and direct match-ups with SoTA methods.\n\n3 . The paper describes the framework and modules clearly and makes it easy to follow and understand. Generally, most of the paper is well written except for some figure captions and details.", "reasons_to_reject": "Although I appreciate the effort and interesting framework in this paper, some weaknesses need to be improved.\n\n1. The number of trials for each experiment is not mentioned in section 4.1 Experimental Setup. It is important to determine the effectiveness and reliability of the proposed method. Please mention the trial number in the Experimental Setup section and the caption of each experimental figure and table.\n\n 2. Since the main contribution of this paper is the proposed framework methods, it lacks of comparison with the reinforcement learning baseline or the large language models baseline (e.g., o1 or deepseek r1). Moreover, it is better to add the human performance as the upper bound because the proposed method uses imitation learning, which uses human demonstrations to capture the core patterns of expert play.\n\n3. The proposed framework only employs the Qwen-2 1.5B as the imitation learning agent base. To demonstrate the effectiveness of this framework, it should adopt at least one more open source model (e.g., llama) as the imitation learning agent base. Furthermore, comparing the closed-source model (e.g., gpt4o) as a specific agent to show the contribution of the imitation learning method. \n\n4. Since the second contribution of this paper is expanding the SC2 evaluation environment,  what is the difference between the proposed evaluation environment and the SC2 evaluation environment? The difference between these two environments is the action space mentioned in the appendix (e.g., the Protoss have 58 possible actions, 475 the Zerg 61, and the Terrans 62)? The detailed comparison should be the main context of the paper.", "questions_to_authors": "For all my major concerns and questions, please see the Reasons To Reject section above. One suggestion is that the author should mention the experiment trial number in the main context and figure caption to help the readers assess the robustness and reliability of this proposed framework. Another suggestion is that it would be better to include human-human performance in this paper to highlight the upper capability boundary of this environment will increase the contribution and the significance of this proposed framework (comparison of human-human corporation and proposed framework).", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a hierarchical framework for strategic reasoning for real-time games such as StarCraft II.  A key idea is to represent planning in text form, and utilize the text-based SCII API.  

The reviewers are generally positive about this work.  One reviewer mentioned several areas where clarity could be improved, and the authors should revise the paper in those aspects.
Decision: Accept