# PALANTIR: An NFV-Based Security-as-a-Service Approach for Automating Threat Mitigation
## 6 Validation
### Quantitative Evaluation
#### 6.3.2. Orchestration of Security Capabilities  
Given the central importance of the life cycle of the SCs by the SO, this was evaluated as an end-to-end process triggered from the SO, testing the three main life cycle operations which are heavily leveraged during the remediation process: (i) instantiation, (ii) reinstantiation, and (iii) configuration. Three of the four currently available SCs were tested, namely, iptnetflow, snort, and suricata. Similarly, the measurements were extracted from a virtualisation environment managed by a K8s cluster, consisting of a control node along with two workers, both enforcing the default deployment setting (i.e., the workers allowed to schedule, the control one tainted). To evaluate the potential performance across different deployment scenarios, the K8s workers were first left running under full resources, and then constrained. Figure 10. Measurements of solving duration compared to constraints model complexity. \begin{table}
\begin{tabular}{c c c c c c c} \hline  & \multicolumn{2}{c}{**Thread Memory (MB)**} & \multicolumn{2}{c}{**Procedure Execution (s)**} & \multicolumn{2}{c}{**Solving Duration (s)**} \\ \hline
**Complexity** & **Mean** & **Median** & **Mean** & **Median** & **Mean** & **Median** \\ \hline
5 & 416.9 & 417 & 0.0311 & 0.0235 & 0.019 & 0.016 \\ \hline
15 & 418.1 & 418.4 & 0.1121 & 0.0941 & 0.097 & 0.086 \\ \hline
25 & 418.6  
It is worth noting that this evaluation assumes a parallelism between the scenario running with full resources and the Edge and Cloud deployment models, given the increased availability of resources for such cases. In addition, the scenario with constrained resources relates to the Lightweight scenario, which features less specifications and is self-hosted. The choice of the specific resources for the workers used in the constrained scenario is motivated by the current availability in the market of lower-end servers with such characteristics for a more modest price (in the range of 2k EUR), presumably more affordable for the microenterprises that would use it as the CPE for the Lightweight deployment model. Specifically, each of the two involved K8s workers were configured as indicated in Table 4, depending on the emulated environment. In addition to this, the OSM instance (acting as the NFV MANO in the architecture) is set as a static VM in another server of the same testbed where the workers run, and it uses two CPUs (one socket with two cores), 8 GB RAM, and 60 GB disks to ensure no disruptions during the operation. The evaluation process was conducted by running 100 times a Python script triggering all three life cycle actions from the SO and measuring its times in a sequential manner, so to ensure proper independence across tests. This process was repeated for each of the three analysed SCs. This all accounts for a total of 900 measurements for each of the 2 considered environments, plotted in the previous figures. It can be appreciated that in Figure 11, the time taken to enact the operations in the full (left) K8s cluster varies with respect to the constrained (right) K8s cluster in that the median time taken by the latter accounts for up to 4.2% of the median time from the original instantiation tests. \begin{table}
\begin{tabular}{l c c c} \hline \hline  & **Sockets** & **Cores** & **RAM (GB)** \\ \hline Workers with full resources & 3 & 4 & 20 \\ \hline Workers with constrained resources & 1 & 4 & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Virtual resources used per emulated deployment scenario. Figure 11: Distribution of instantiation times for the three devised SCs, comparing full (**left**) and constrained (**right**) deployments. Likewise, Figure 12 shows similar differences when comparing the median times taken by reinstantiation, introducing a difference across such values up to 2.2%.