## 6 Open Directions and Challenges
### Challenges in Cyber Defense Decision-Making  
Network defense decision-making requires in-depth study of problem characteristics in combination with real scenarios, so as to provide a basis for more convenient simulation experiments and theoretical analysis, as well as better evaluation of defense decision-making effects. The challenges in network defense decision-making are as follows. **Task model:** In order to accurately characterize the characteristics of cyber attack and defense interactions, the researchers proposed FilpIt [105; 106], Hide-and- Seek [107; 108; 109; 110], two-person zero-sum game [55] and other task models to analyze the decision-making behavior of both attacker and defender. The above models have abstracted the characteristics of network attack and defense, such as concealability and dynamics, but it is still difficult to provide guidance for the decision-making process due to the theoretical reasons as follows: (1) The design of environment, action and state of the existing models is abstract and lacks the mapping with the real world. (2) Most network attack and defense games are abstracted as two-person zero-sum games, whose design of reward and ending mechanism cannot accurately reflect the decision-making preferences of both attacker and defender. Therefore, to conduct a more in-depth analysis on the characteristics of attack and defense game in actual task scenarios such as Pentesting and virus infection is necessary. **Simulation platform:** Previous studies mainly verify the effectiveness of network attack and defense decision-making algorithm by MulVAL [111], NS3 [117], Petri net [116] real data playback or using virtual machine network. In Table 3, the relevant work of the existing training and testing platform is summarized, which is divided into three indexes of high, middle and low to measure the convenience, fidelity and expansibility of each network training and testing platform. 1. [label=(0)]
2. Convenience: How easy the platform is to use, which is evaluated on the basis of the size of the package, the platform on which it is running, the language in which it is compiled, and how it is installed. 3. Fidelity: Closeness to the real world, which is mainly evaluated from action design, network environment, evaluation indexes, etc. 4. Expansibility: The difficulty of secondary development, which is evaluated based on the open source degree of the platform and the difficulty of the experiment of researchers. Obviously, using NS3, real data playback and other methods to build the training test environment will bring a lot of unnecessary details in terms of decision-making, such as raw data preprocessing or traffic packet parsing. As a result, the processing complexity of the algorithm is increased. However, Petri net or MulVAL and other methods to build the training testing environment need a lot of pre-defined work, which is difficult to reflect the dynamic of network attack and defense. Therefore, it has become a current development trend for researchers to use multi-agent methods to build training and testing platforms for attack and defense, such as the Network Attack Simulator [59] by [PERSON] and CyberBattleSim [120] by Microsoft. However, these training test platforms are still developing, \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Network Implementation** & **Model** & **Literature** & **Convenience** & **Fidelity** & **Expansibility** \\ \hline \multirow{3}{*}{Network Simulation} & MulVAL [111] & [112; 113; 114; 115] & Middle & Low & Low \\  & Petri net [116] & [58] & Low & High & High \\  & NS3 [117] & [118] & Middle & High & Middle \\  & Mininet [119] & [65] & Middle & Middle & Middle \\ \hline \multirow{2}{*}{Testing platform} & NASim [59] & [48] & High & Middle & High \\  & CyberBattleSim [120] & - & High & Middle & High \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of Current Network Training and Testing Platform. which are difficult to define and expand the elements of attacking and defending actions, states and tasks flexibly.