## 7 Evaluation
### Graph-Based Model Composition  
For the purpose of evaluating the proposed composition method, it might have been convenient to use one of the existing metrics for analyzing process discovery algorithms, such as replay fitness, simplicity, precision and generalization [79]. However, in process composition, we focus mainly on the allowed behavior of the process instead of analyzing the graphical layout of the model. Therefore, we determine the rate to which the generated BPMN diagram can satisfy the set of workflow traces generated by solving the constraint satisfaction problem.  
In order to do so, we use the concept of log-based process model verification, which consists of generating a synthetic set of traces based on the resulting diagram and comparing it to the original log. A model can be considered perfectly composed if the original log is sufficiently complete (see Definition 1) with respect to this model. Thus, the generated model should allow one to execute all the sequences represented by original traces and no more. To verify this ability, we propose a metric called composition accuracy. It combines the following measures:  
* model fitness: the percentage of traces from the original log, which were generated based on the composed model,
* execution precision: the percentage of generated workflow traces that are allowed in the original log. \[CA=\textit{model\_fitness}\times\textit{execution\_precision}\] (9)  
The metric can be calculated as defined in Formula (9). In order to illustrate its use, we provide an elementary example. Let \(W_{C}\) be a complete workflow log generated based on a declarative specification (Formula (10)).  
\[W_{C}=\{\{A,B,C\},\{B,A,C\},\{A,B,C,D\},\{B,A,C,D\}\}. \tag{10}\]  
Let us assume that a BPMN model was composed based on \(\log W_{C}\). The set of traces generated from the resulting model is a synthetic log \(W_{S}\) (Formula (11)).  
\[W_{S}=\{\{A,B,C,D\},\{B,A,C,D\},\{A,C,B,D\}\}. \tag{11}\]  
As can be clearly seen, two logs significantly differ. First, the generated model allows one to reproduce only half of the original traces since traces without activity \(D\) do not occur in \(W_{S}\). The model fitness is then equal to 50%. Secondly, \(\log W_{C}\) does not contain sequence \(\{A,C,B,D\}\). In this case, the composed model allows for more behavior than the original log. This leads to an execution precision equal to 66.67%. Multiplying these two metrics results in a composition accuracy equal to 33.34%.  
In order to verify the ability of our approach to compose BPMN models based on a synthetic workflow log, we compare it to process mining algorithms that were briefly reviewed in Table 7. At the first step, we have excluded Alpha Miner because of its inability to discover loops [68]. Figure 10 presents the results of comparing the activity graph-based composition described in Section 6.2 to three process discovery algorithms, namely: ILP Miner, Inductive Miner and Heuristic Miner. All the mining algorithms were executed in the ProM 6 environment [59], with a minimal noise threshold and default parameters for the others.  
The results show that there is no explicit correlation between log-based complexity and the composition accuracy. However, one may observe that the models where process mining algorithms achieved less accurate results are characterized by higher looping depth (see Table 9). Therefore, we have grouped the results by looping depth and presented the average composition accuracy for each value in Figure 11.  
Figure 10: Comparison of applying different methods for BPMN diagram generation.