## 3 Method
### Overview  
Our framework, illustrated in Figure 1, is designed as a workflow that emulates human reasoning, with decision-making and analysis at its core, and the execution of decisions facilitated by external tools. This design reflects the typical cognitive processes and resource utilization involved in solving a CTF challenge. Based on this insight, we divide the tasks assigned to agents into two categories: the first simulates human reasoning, focusing on analyzing and monitoring the information generated throughout the problem-solving process and proposing potential actions; the second simulates the execution of actions using various external tools to process the available information and return the results to the reasoning agents.  
Enhancing agents with the ability to invoke external tools proves to be an effective approach in solving CTF challenges, as many such tasks involve rigorous and complex computational procedures that are prone to failure due to hallucinations. We equip agents with four primary external capabilities: (1) performing basic interactive operations via terminal control; (2) generating and executing code to accomplish core problem-solving tasks; (3) leveraging web browsing to expand the range of accessible tools; (4) summarizing and extracting content from attachments.  
For both human players and intelligent agents, establishing a clear analytical backbone throughout the problem-solving process is essential. For language models in particular,  
Figure 1: An overview of our multi-agent CTF automation framework. The **LLM** module orchestrates a set of expert agents, including analyst (for collecting and analyzing the problem content), decision maker (for invoking suitable tool and generating formatted command), shell operator, programmer, web explorer, reference checker, and serves as the core pipeline for executing the “think-decide-act-reflect” loop. The **Knowledge** module aggregates information specific to the given CTF problem, including challenge descriptions, clues, hints, and other relevant context. The **Computer Host** module provides an execution environment for tool invocation, enabling agents to interact with it through various modalities.  
intermediate interactions resulting from tool invocation can consume a substantial amount of context length, and the progressively expanding analysis can obscure the main logic. Therefore, adopting a multi-agent architecture becomes a natural solution, allowing different agents to handle heterogeneous tasks while maintaining control over dialogue length and context relevance.  
Figure 2 presents an example of our agents solving a classical cipher challenge. The agents first identify the cipher as Base64 encoding and generate a script to decode it, resulting in another ciphertext. They then determine that the new ciphertext likely uses a substitution cipher, hypothesize that it may be encrypted using the Vigenere cipher, and utilize a web-based decryption tool to recover the plaintext question. Then the answer to the question is given, and the test is passed. In this process, the agent directly solves simpler ciphers through programming, while delegating more complex or computation-intensive tasks to efficient external tools.  
Figure 2: An example of solving a CTF challenge. Rounded rectangles with different background colors represent distinct agents (_Analyst_, _Programmer_, and _Web Explorer_). Each iteration begins with the _Analyst_ and leads to a specific action. The “Machine” field denotes the formal instruction generated by the _Decision Maker_ based on the Analyst’s reasoning. The “Result” field is the new information propagated to the next iteration. Chat messages are arranged in Markdown format.