# A Data-Driven Approach to Discovering Process Choreography
## 5 Experimentation
### Experiment Results
Table 5 shows the validation of the quality of the intra-organizational process models discovered, which were recovered from the behavior identified in the event logs through implementing the Split-miner algorithm. The level of quality of the recovered intra-organizational process models is of great importance in the proposed approach, since these models are the basis for discovering inter-organizational processes. Table 5 shows the average value of the evaluation metrics per scenario based on the intra-organizational process models discovered in each scenario. For the _precision_ metric, a value between 0.85 and 1.00 was obtained, considering the six scenarios. For the _recall_ metric, a value of 1.00 was obtained in all the scenarios evaluated. Values of 0.30 and 0.54 were obtained for the _generalization_ metric in the scenarios _transfer of goods_ and _manufacturing process_; the evaluation for the rest of the scenarios was between 0.79 and 0.96 for this metric. Overall, these results indicate that the intra-organizational models replicated all behavior from event logs and that the models accounted for 5% of behaviors not included in the event logs. However, it is observed that there was a large number of infrequent events, according to the values obtained for the _generalization_ metric. On the other hand, Table 6 presents the evaluation of the quality level of the IOBPs discovered through the metrics of precision, recall, and generalization. In the experiment, 21 organizations participating in 20 peer collaborations were identified, derived from the event logs analyzed. Considering a general evaluation, a value of 0.94 was obtained for the _precision_ metric, 0.99 for the _recall_ measure, and a value of 0.63 for the _generalization_ indicator. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Scenario** & **RP** & **RR** & **FsR** \\ \hline Air quality system & 1 & 1 & 1 \\ Healthcare & 0.62 & 0.50 & 0.54 \\ Travel agency & 1 & 1 & 1 \\ Purchase order & 0.83 & 1 & 0.90 \\ Transfer of goods & 0.83 & 0.88 & 0.83 \\ Manufacturing process & 0.91 & 1 & 0.92 \\ Average & 0.86 & 0.89 & 0.86 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average evaluation of the identification of events at the collaboration level. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Intra-Organizational Business Process** & **Precision** & **Recall** & **Generalization** \\ \hline Air quality system & 1 & 1 & 0.96 \\ Healthcare & 0.98 & 1 & 0.87 \\ Travel agency & 0.85 & 1 & 0.89 \\ Purchase order & 1 & 1 & 0.79 \\ Transfer of goods & 1 & 1 & 0.30 \\ Manufacturing process & 0.88 & 1 & 0.54 \\ Average & 0.95 & 1 & 0.72 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Quality assessment of discovered intra-organizational business process models. At that level, the results obtained are acceptable and indicate that the inter-organizational process models discovered were capable of reflecting 99% of the behavior found in the merged collaborative event logs. Furthermore, the discovered models only reflected 6% of behaviors not seen in the event logs. However, low values remained for the _generalization_ metric, which denotes infrequent activities in the event log. It is essential to mention that this metric is only informative for knowing the conformation of the event log and does not express a performance value for the discovered model. Due to the nature of business processes, it is common for there to be infrequent activities or behaviors in an event log. Figure 2 shows a comparison between the reference IOBP model (see Figure 2a) and the IOBP model discovered (see Figure 2b) for the scenario _Travel Agency_. The figure shows that the process logic, tasks, and gateways discovered coincided with the behavior contained in the reference IOBP. The process choreography deployed through the interaction of messages between pools, represented by the message flow connectors, was similar among the reference and discovered models, with the logic of the matching process and the logic and sequence of the process choreography.