# A Data-Driven Approach to Discovering Process Choreography
## 5 Experimentation
### Experiment Results  
This section presents a detailed evaluation (for each scenario) of implementing the proposed methodology for discovering process choreography by identifying message-type events. The performance achieved by the implementation of the approach was measured using the metrics _relationship Precision (PR)_ (Equation (3)), _relationship Recall (RR)_ (Equation (4)), and _F-score of the relationship (FsR)_ (Equation (5)). Moreover, a quantitative evaluation of the quality of the intra-organizational and inter-organizational process models discovered by the proposed approach is presented using the _precision_, _Recall_, and _Generalization_ metrics. Table 3 presents the values obtained for each evaluation metric of the process choreography discovered for each inter-organizational scenario. The second column shows the pairs of participants for which a message exchange was identified. Columns 3, 4, and 5 display the metrics _PR_, _RR_, and _FsR_ achieved in the evaluation of the discovered choreography. In addition, Table 3 shows the \(U_{t}\) and \(U_{a}\) thresholds defined for each scenario. The sensitivity of each threshold can be interpreted as follows: If the value of the threshold \begin{table}
\begin{tabular}{l l c c c c} \hline \hline
**Scenario** & **Participants** & **Collaborations** & **Cases** & **Events** & **Unique Events** \\ \hline Air quality system & IoT Air Quality Monitor (IAQM) & 2 & 9180 & 511,049 & 24 \\  & Repository Management Service (RMS) & & 9180 & 215,651 & 8 \\  & System Access Service (SAS) & & 9180 & 64,260 & 7 \\ Healthcare & Patient & 4 & 100 & 250 & 4 \\  & Laboratory & & 100 & 300 & 3 \\  & Hospital & & 50 & 200 & 5 \\  & Gynecologist & & 100 & 700 & 9 \\ Travel Agency & Travel Agency & 1 & 100 & 869 & 5 \\  & Customer & & 100 & 607 & 5 \\ Purchase order & M-Repair & 1 & 100 & 714 & 19 \\  & M-Parts & & 100 & 686 & 18 \\ Transfer of goods & Shipper & 4 & 2 & 24 & 12 \\  & Sender and Buyer & & 2 & 20 & 10 \\  & Consigner & & 2 & 40 & 20 \\  & Carrier & & 2 & 20 & 10 \\ Manufacturing process & Bulkbuyer & 8 & 5 & 60 & 4 \\  & Middleman & & 5 & 56 & 4 \\  & Manufacturer & & 5 & 202 & 20 \\  & Supplier A & & 5 & 78 & 8 \\  & Supplier B & & 5 & 19 & 4 \\  & Special Carrier & & 5 & 39 & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Characteristics of the event log of each inter-organizational scenario. \(U_{t}\) is close to 0, a greater number of cases from the event log are analyzed to find the message-type events. On the contrary, if the value of \(U_{t}\) is close to 1, the method analyzes fewer cases, reducing the search space. The \(U_{a}\) threshold is highly sensitive, because a change in this parameter's value will impact the values achieved in the RP, RR, and FsR metrics. So, if the threshold \(U_{a}\) is decreased close to 0, this will allow more relevant relationships to be recovered, making the value of the RP metric high. However, this will cause more non-relevant relationships to be recovered, which will harm (reduce) the RR metric. Otherwise, if the \(U_{a}\) threshold is increased close to 1, this can cause the relationships found to be true and relevant relationships. Therefore, the RR metric would increase, while the RP metric would decrease, because some relevant relationships would not be recovered due to a low value in the similarity measure. In our experiment, the value of \(U_{a}\) was assigned by maximizing the FsR metric, in order to recover the greatest number of relevant relationships and the least number of non-relevant relationships, since FsR represents the weighting of the RP and RR metrics. Moreover, the time costs required for the identification of the message tasks were very acceptable.