# An Interactive Method for Detection of Process Activity Executions from IoT Data +
## 5 Proof of Concept Evaluation
### Results  
Table 5 summarizes the relevance determined by the analyst for all the CPS components of the smart factory (cf. Step 2, Section 4). Seven relevant time frames were identified from the IoT data set based on the visualization of changes in sensor and actuator values within the individual CPS components. The corresponding start and end time stamps (absolute time stamps) are reported in the table. The analyst determined that the vacuum gripper robot (VGR_1) was relevant in parts 1, 3, 5, and 7, the high-bay warehouse (HBW_1) in parts 3 and 7, and the oven (OV_1) in parts 2 and 6. Other components were considered not relevant in any time frame due to the absence of changes (MM_1, WT_1, SM_1) or insignificance of changes (EC_1) in the corresponding data. Figure 19: Plot of IoT data set used in evaluation. Figure 20: Process model (in BPMN 2.0) used for evaluation. Figure 21 illustrates the result of the manual annotation done by the analyst (activity boundaries in green), complemented with the annotation directly derived from the process event log created by the BPM system (activity boundaries in red). The figure also shows suggested abbreviated labels for the activities (in green suggested by the analyst, in red suggested by the BPM system, and in black where the analyst and BPM system match) and distinction into process instances. The identification of the type of activity by the analyst almost completely matches the ground truth. For the second activity executed, the process analyst was uncertain about the semantics of this activity. A sequence of movements of the vacuum gripper including dropping and picking up the workpiece again was observable in the data. However, no other component was active in between. Here the actual intention is that the color of the workpiece is determined with the help of the color sensor at the delivery and pickup station connected with the VGR_1 component. This activity does only refer to reading a specific sensor, which is constantly emitting data. Thus, this explicit sensor data retrieval cannot be observed in the IoT data. Additionally, we see deviations in the timestamps related to the start and end of activities generated by the BPM system and identified by the process analyst. The process analyst focuses on sensor/actuator patterns as the only indicators for activity boundaries. However, the actual implementation and execution of the specific activity by the BPM system refers to a call to a web service [17]. This invocation might involve additional computations, requests to the other applications, or other forms of communication, which might introduce delay. Thus, there may be mismatches emerging between the recorded start/end of an activity and its manifestation in the IoT data. Examples here are the implementation of (1) the \(Burn\) activity, which includes a short period of waiting before moving the workpiece into the oven, and (2) the _Store Workpiece_ activity, which is considered to have ended once the container has been loaded into the warehouse, although the loading crane is moving to its initial position afterward. Based on the similarities of identified activity sequences, the process analyst was able to successfully identify the execution of two process instances belonging to the same process. From these instances, the analyst was able to recreate the process model accordingly, which--apart from the uncertain second activity--corresponds to the model used for execution (cf. Figure 20). Therefore, we conclude that the detection of activities and the whole process following the developed method was achieved successfully.