# Physics-Informed Graph Neural Networks for Attack Path Prediction
## 3 Results
### Sensitivity Analysis  
We conducted a sensitivity analysis with the objective of measuring the importance of each node feature and edge type in the architecture prediction. To estimate this value, we used the SHAP algorithm [105] (see Appendix F). Figures 7-9 show the results of our analysis for \(\mathcal{M}_{1}\left(\Psi=1\right)\), with the PIGNN and \(\mathcal{M}_{2}\) and \(\mathcal{M}_{3}\) the start-node and exit-node prediction models, respectively. Features and edge types are ordered on the Y-axis of each plot, in descending order. The dot colors indicate the value of the input (blue for values \(<0\), red for values \(\geq 0\)), and the dot location on the X-axis indicates the influence of the value on the prediction. For \(\mathcal{M}_{1}\) (see Figure 7), **(1)** Open edges with low values--corresponding to edges that do not have a corresponding network route--are very influential in preventing concurrent edges from being included in the predicted attack path. It appears that the PIGNN learned this association, which is coherent with domain knowledge, as non-routed paths cannot be exploited by an attacker. **(2)** the AdminTo, ExecuteDCOM, GenericAll, CanRDP, and HasSession edges are also positively correlated with the prediction, meaning that such edges are perceived as likely to be part of the attack path by the neural network. This is also expected, as these edges correspond to easily exploitable lateral movement techniques, and as discussed in Section 2.1, the AdminTo and HasSession features are highly correlated with ground-truth attack paths. From the feature SHAP values, we can observe that **(3)** the Start Node property exhibits a large influence on the neural network depending on its value. This could be an indicator that the feature is used by the network to decide whether to include or exclude the nodes in the path. This makes sense, as any node with this attribute set is necessarily included in the path. **(4)** Most of the other features are centered around 0 with relatively narrow amplitude. Overall, these results seem to indicate that the network focuses largely on the edge types rather than node attributes, with the exception of the entry node. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Model** & **Mean** & **Std.** & **Fold 1** & **Fold 2** & **Fold 3** & **Fold 4** & **Fold 5** & **Validation** & **Threshold** \\ \hline \hline \multicolumn{10}{c}{**ROC-AUC Scores**} \\ \hline \(\mathcal{M}_{1}\left(\Psi=1\right)\) & 0.9332 & \(\pm\)0.0081 & 0.9300 & 0.9329 & 0.9201 & 0.9429 & 0.9402 & 0.9533 & 0.7189 \\ \(\mathcal{M}_{1}\left(\Psi=0\right)\) & 0.8318 & \(\pm\)0.0018 & 0.8296 & 0.8301 & 0.8319 & 0.8344 & 0.8328 & 0.8382 & 0.7311 \\ \(\mathcal{M}_{2}\) & 0.8340 & \(\pm\)0.1128 & 0.9490 & 0.9407 & 0.8276 & 0.6374 & 0.8328 & 0.9037 & 0.0162 \\ \(\mathcal{M}_{3}\) & 0.8312 & \(\pm\)0.1379 & 0.8335 & 0.9239 & 0.9155 & 0.5638 & 0.9195 & 0.8081 & 0.0247 \\ \hline \hline \multicolumn{10}{c}{**FI Scores**} \\ \hline \(\mathcal{M}_{1}\left(\Psi=1\right)\) & 0.9177 & \(\pm\)0.0082 & 0.9178 & 0.9326 & 0.9080 & 0.9170 & 0.9133 & 0.9308 & 0.7189 \\ \(\mathcal{M}_{1}\left(\Psi=0\right)\) & 0.8166 & \(\pm\)0.0088 & 0.8288 & 0.8136 & 0.8056 & 0.8250 & 0.8101 & 0.8191 & 0.7311 \\ \(\mathcal{M}_{2}\) & 0.9154 & \(\pm\)0.0928 & 0.9777 & 0.9640 & 0.9126 & 0.7368 & 0.9857 & 0.9780 & 0.0162 \\ \(\mathcal{M}_{3}\) & 0.8644 & \(\pm\)0.0964 & 0.8254 & 0.9370 & 0.9330 & 0.6913 & 0.9352 & 0.8214 & 0.0247 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance benchmark of the path prediction architecture with and without physics-informed loss \(\mathcal{L}_{\text{pinn}}\). \(\mathcal{M}_{1}\left(\Psi=1\right)\) uses the physics-informed loss, and \(\mathcal{M}_{1}\left(\Psi=0\right)\) does not. \(\mathcal{M}_{2}\) and \(\mathcal{M}_{3}\) are the start-node and end-node predictors, respectively; the thresholds obtained using Youden’s J index of each model were used to compute F1 scores.