# CvAMoS--Event Abstraction Using Contextual Information
## 5 Evaluation
### Evaluation on Synthetic Datasets
CvAMoS has longer motifs and consequently _motif length_ as well. _Distance_ and _quorum_ are relaxed too. The two additional runs (R4, R5) are only applied to CvAMoS. The objective of these is to verify that motifs with attributes are correctly identified. The settings are shown in Table 3. The _motif length_ is set to the length of the motif to be recognized. In this case, the _attribute distance_ is crucial. The _activity distance_ and the _attribute distance_ are both set to zero in R4 to verify the precision, and they are increased in R5 to verify the behavior of the algorithm in handling noise. The outputs of the runs are evaluated according to the metrics defined above, i.e., Q1, Q2, as well as the time. The first point to discuss regards the evaluation of the first three runs: R1, R2, and R3. The main difference in the results of the two algorithms in these three configurations lies in the precision in recognizing the given pattern. The values of Q1 are shown in Figure 2a: considering the three runs, with a total of 42 executions for each algorithm (8 + 6 patterns * 3 configurations), Q1 holds only 7 times for LPM, but 32 times for CvAMoS. Therefore, LPM was able to derive a model representing the given pattern only in a limited number of cases with respect to the other algorithm. The performance of Q2 instead is affected by the number of outputs to produce for LPM and the activity distance for CvAMoS (if allows us to return motifs with \(d\) non-matching activities). However, the combination of 0 for Q1 and 1 for Q2 means that the algorithm is not only unable to identify the given pattern, but identifies more than one non-fitting trace. Figure 2b plots the values of QR. The third metric evaluated is the computation time. There is no evidence that an increase/decrease in the computation time influences the performance of the algorithms. The time, in a logarithmic scale, is shown in Figure 3. For both algorithms, the computation time rises for each run, partly due to the considerable expansion of the number of candidate motifs/models. Although a greater number of candidates is synonymous with greater precision in the results for CvAMoS, this is not observed for LPM. The second point to discuss regards the evaluation of R4 and R5. In this case, only CvAMoS is applied, and the capability of the algorithm to deal with contextual variables is tested. The result of the runs is shown in Table 4, which reports the QR values for each application. In R4, four cases returned a perfect result, meaning that the algorithm was able to identify instances of the same pattern, but with different attributes, and that it did not generate non-fitting patterns (QR is 1). In four cases, no motif was found (QR is 0). However, this last information does not refer only to this case; in fact, the algorithm was never able to identify, even in the previous runs, the patterns P4 and P6, due to their variability. In the last run, R5 has better results in the identification of the patterns, as the algorithm failed in only two cases (still P4 and P6). However, in all the remaining cases, the \begin{table}
\begin{tabular}{c l l l} \hline \hline  & & **R4** & **R5** \\ \hline \multirow{4}{*}{CvAMoS} & Motif length & 6 & 6 \\  & Quorum & 0.7 & 0.7 \\ \cline{1-1}  & Max activity distance & 0 & 1 \\ \cline{1-1}  & Max attribute distance & 0 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Algorithm configuration for the evaluation with context. value of QR is 0.5, as Q2 is always true, meaning that the algorithm was able to identify the target patterns, but also generated non-fitting sequences. To summarize, the results obtained using a synthetic dataset show that:  
* Considering the first three runs (R1-R2-R3), in R1, \(\text{CvAMoS}\) showed better performances in terms of time and QR compared to LPM. In fact, it recognized the highest amount of patterns (QR is 0.48) in a short period of time (440 ms). In the same run, LPM showed the highest QR score, that is 0.26.