# Using Graph Embedding Techniques in Process-Oriented Case-Based Reasoning
## 6 Experimental Evaluation
### Experimental Results
For instance, sGEM can filter with _fs_ = 80 in less time and with higher ultimate quality than FBM can filter with _fs_ = 10 to obtain the results for \(k\) = 10. Overall, H1 is partly confirmed due to the similar results of the comparison between sGEM and EBM. None of the different measures clearly outperforms the other. However, sGEM can improve the overall retrieval results under certain conditions (_fs_, \(k\), variant, etc. ), which have to be tested for the given scenario. The second experiment aims to answer hypothesis H2 and examines the degree to which the variants of sGEM and sGMN as well as EBM and FBM can approximate the ground-truth graph similarities (see Table 2). We also include a configuration of A*M with an adjusted parameter value of MaxPMS (see Section 2.2) in the experiment. The value for MaxPMS is chosen in such a way that the retrieval time of A*M is similar to that of the sGMN variants. Aligning the retrieval times of A*M and sGMN enables a fair comparison of the resulting MAE and correctness. All reported times are measured in milliseconds. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Domain** & \multicolumn{3}{c}{**CB-I**} & \multicolumn{3}{c}{**CB-II**} \\
**Retriever** & **MAE** & **Correctness** & **Time** & **MAE** & **Correctness** & **Time** \\ \hline sGEM & 0.158 & 0.017 & 1.3 & 0.337 & 0.331 & 1.1 \\ sGEM\({}_{\text{tree}}\) & 0.219 & 0.053 & **0.9** & 0.323 & 0.310 & **0.9** \\ sGMN & 0.033 & 0.287 & 1025.5 & 0.034 & 0.583 & 2697.8 \\ sGMN\({}_{\text{tree}}\) & 0.039 & 0.322 & 2115.1 & 0.026 & 0.724 & 3508.1 \\ sGMN\({}_{\text{const}}\) & **0.029** & 0.330 & 1051.9 & 0.033 & 0.583 & 2643.7 \\ sGMN\({}_{\text{tree,const}}\) & 0.037 & 0.327 & 2118.3 & **0.024** & 0.732 & 3403.8 \\ FBM & 0.193 & 0.598 & 482.7 & 0.199 & 0.584 & 335.4 \\ EBM & 0.380 & 0.224 & 1.2 & 0.397 & 0.006 & 1.1 \\ A*M & 0.062 & **0.669** & 1265.5 & 0.041 & **0.824** & 3801.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation results of the A* approximation experiment. For CB-I, sGMN\({}_{\text{const}}\) has the lowest MAE value and A*M has the highest correctness value. The sGEM variants report relatively high MAE values and relatively low values of correctness. The MAE and the correctness values of all sGMN variants are in a similar range, while all outperform the MAE of A*M but are outperformed by A*M in terms of correctness. FBM achieves a high level of correctness but lags in terms of MAE. EBM shows high values of MAE and low values of correctness. When comparing the results of CB-I and CB-II, it becomes apparent that sGMN\({}_{\text{tree,const}}\) has the lowest MAE and A*M again has the highest value of correctness. The results of the other measures are similar to the results for CB-I. However, it is noticeable that the gap between the sGMN variants and A*M in terms of correctness is smaller. This leads to the assumption that the suitability of sGMN increases with more complex cases. The fact that the variants of sGMN outperform A*M in terms of MAE is even more remarkable when considering that the neural network learns to assess the similarity of graphs without knowing the original algorithmic context, e.g., similarities of semantic descriptions or node and edge mappings. Additionally, this experiment shows that FBM and EBM are not suitable for generating similarities that are close to the ground-truth similarities. The reason for this could be the inadequate processing of semantic annotations and the workflow structure. Thus, we partly accept H2 since the sGMN variants consistently outperform the MAE values of A*M but do not report higher values of correctness.