## 4 Experiment
### Results and Analysis
#### 4.4.4 PDSSS System Analysis  
We have researched and developed a Pareto Defense Strategy Selection Simulator (PDSSS) system for assisting network administrators on decision-making, specifically, on defense strategy selection. The objective of the system is to develop a system that addresses the multi-criteria decision making using Pareto efficiency. The purpose is to assist network security managers in choosing a defense strategy to mitigate the damage to the business by an attack. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**Running Times (in Seconds)**} \\ \cline{2-4}  & **Mean** & **Std. Dev.** & **99\% C.I.** \\ \hline Pareto-Q-learning & 2.5036 & 0.2118 & \(\pm\)0.0772 \\ Linear scalarization & 0.0033 & 0.00039 & \(\pm\)0.00014 \\ STOM & 0.0065 & 0.00005 & \(\pm\)0.00002 \\ GUESS & 0.0079 & 0.00060 & \(\pm\)0.00022 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Running time analysis. Figure 8: Distribution of \(f^{(1)}/f^{(2)}\) ratio. Our system is developed by specifically focusing on one specific type of attack, the network flooding attack. However, with slight changes the system should work with any attack that has multiple objectives. The goals, tasks, and solutions of PDSSS are listed below in Table 4. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \# & **Goals** & **Tasks** & **Sub-Task** & **Solutions** \\ \hline \multirow{4}{*}{1} & \multirow{4}{*}{System Initialization} & \multirow{4}{*}{Topology configuration} & \multicolumn{2}{c}{Node attributes setting} & Data import/Manual \\  & & & & user import \\ \cline{3-5}  & & & Connections/Edge & & \\ \cline{3-5}  & & & configuration & Data import/Manual \\ \cline{3-5}  & & & Sparsity & user import \\ \cline{3-5}  & & & Goal definition & Service cost setting & \\ \cline{3-5}  & & & Agent initial & Attacker point setting & \\ \cline{3-5}  & & & configuration & Defender point settings & \\ \hline \multirow{4}{*}{2} & \multirow{4}{*}{Simulate the agent actions} & \multicolumn{2}{c}{Attacker behavior} & Randomized behavior \\ \cline{3-5}  & & & Static selection & Pareto Set method \\ \cline{3-5}  & & & Machines Learning & Pareto-Q-learning \\ \cline{3-5}  & & & Machine Learning & \\ \cline{3-5}  & & & Comparison & \\ \hline \multirow{3}{*}{3} & \multirow{3}{*}{Output} & Suggested strategies & For each selection method & UI: List of Strategies \\ \cline{3-5}  & & & Final State & Topology of the network & UI: Graph \\ \cline{3-5}  & & & UI: Brief text description \\ \hline \hline \end{tabular}
\end{table} 
Table 4: System tasks and requirements. Figure 9: Distribution of algorithm running times. The GUI (Graphical User Interface) layout is visible in the wireframe in Figure 10, which consists of five sections:  
* The configuration panel Here the user can choose the input configuration of the model. There are three models, each of which has a different set of goal functions. * The goal function before the application of optimization Here the user can see the solution space for the goal functions before we implement the algorithm (i.e., the results from selecting a strategy as defined by the model). * The goal function after the application of optimization Here the user can see the solution space for the goal functions after we implement the algorithm (i.e., the results from selecting a strategy as defined by the model). * The results Here the user sees a summary of what the optimization algorithm has done for a single step. * The command buttons Here the user interacts with the GUI by generating a new model with the given configuration or by loading an existing model from a csv (Comma-Separated Values) file. In order to validate the proposed models using simulated results, we use the SNAP email-EU-core network dataset to extract the network topology. The network dataset in Table 5 contains email data from a large European research institution. The edge (u, v) represents the existence of an email communication between two users.