# Combining Supervised and Reinforcement Learning to Build a Generic Defensive Cyber Agent
## 4 Performance Evaluation
### Varying the Network Topology
#### 4.2.1 Topology 2 Results
BL and RED demonstrate the worst performance, as also shown in Table 10. It is important to note here that VAE-V demonstrates statistically better mean rewards compared to RED against _red_meander_. When agents' performance for 50-step episodes using Topology 2 with the topology used for agents' training, the performance of blue agents trained through the proposed framework is improved against the _red_meander_ agent but is slightly deteriorated against the _b_line_ agent. Most of the HAs performance is slightly deteriorated in this case. should not be significantly impacted with this change. Additionally, the host that both red agents use to access the network is now closer to their target, so the red agents may not perform poorly or could even perform better. Topology 3 is more challenging for the trained blue agents because the red agents now need fewer actions to compromise the operational server. Therefore, Topology 3 can provide more insights into the generalizability of the trained blue agents with regard to network topology variations. Figure 13 presents the performance of various blue agents against _b_line_ and _red_meander_ over Topology 3. MLP shows a statistically significantly higher mean reward compared to all other trained blue agents evaluated in this study. Notably, all blue agents trained using the proposed framework demonstrate a statistically significantly better mean reward compared to BL, the blue agent specifically trained for _b_line_. All variants of HAs show a higher mean reward compared to BL and RED; however, this difference is not statistically significant. Against _red_meander_, SVM and VAE-V exhibit statistically significantly better performance compared to RED, the blue agent specifically trained for _red_meander_. Furthermore, SVM shows statistically significantly better performance compared to all other evaluated blue agents in this scenario, while MLP demonstrates a comparatively poor performance, albeit slightly better than that of the blue agents trained solely via RL. Overall, against both red agents, the blue agents trained through the proposed framework showed a better mean reward compared to all other evaluated blue agents. Two out of three blue agents trained with the proposed framework achieve a statistically significantly higher mean reward compared to pure RL blue agents. Table 11 shows the total mean reward exhibited by the blue agents while considering both red agents. SVM exhibits the highest mean reward (shown in bold face), followed by VAE-E and MLP. Comparing Figure 7 with Figure 13 reveals the following about the trained blue agents' performance against the red agents:  
* When the attacking red agent is _b_line_, BL and VAE-V demonstrate similar performance in both topologies. SVM, VAE-V, RED, and all HAs demonstrate better performance in Topology 3. * When the attacking red agent is _red_meander_, all blue agents' performance deteriorates in Topology 3. Figure 14 shows the mean reward of the blue agents against both red agents over a 50-step episode. Generally, the trends in Figure 14 are similar to those in Figure 13. The agents trained using the proposed framework exhibit better mean rewards, with instances of statistically significantly better performance compared to a blue agent specifically trained against a particular red agent. Similarly, the trends in Table 12 mostly mirror those in Table 12, where SVM demonstrates the highest total mean reward (shown in bold face), followed by VAE-V and MLP. Variants of HAs show the lowest total mean reward when considering their performance against both red agents. Comparing Figure 14 with Figure 8  
Figure 12: Network Topology 3.  
we evaluated blue agents on network topologies different from those used for training (the total number of nodes in the network remained the same, but the number of nodes in each subnet was altered). Across these different network topologies, the blue agents trained with the proposed framework demonstrate statistically significantly better performance compared to other evaluated blue agents.