# An Interactive Method for Detection of Process Activity Executions from IoT Data +
## 5 Proof of Concept Evaluation
### Observations  
During analysis, the process analyst took note of some relevant observations to document difficulties and challenges:  
* The relevance of CPS components and time frames was easy to determine given knowledge about the characteristics of sensors and actuators (cf. Section 4.3). However, determining the size of a time frame based on a component being active/inactive may already result in a very fine-grained segmentation of the data set.
* Activities executed by CPS components that involve only a small number of sensors and actuators and that only offer one or two different activity types (e.g., the oven) were easy to identify. Here the analyst was able to identify the start and end based on simple change patterns (cf. Section 4.4.1) within the values of a sensor or actuator with high accuracy compared to the ground truth. This is the case, for example, for the _Burn_ activity by OV_1. However, as mentioned in Section 5.2 there might be deviations in the precision of identified time stamps due to the implementation of an activity not directly manifesting itself in the IoT data.
* Activities executed by CPS components that involve multiple sensors and actuators and depending on domain knowledge regarding the change patterns (e.g., referring to the target position of the vacuum gripper robot) were more difficult to identify. Here it was not always obvious which combinations of change patterns among one or multiple sensors and actuators indicate the start or end, and which activity was identified. Moreover, finding the right level of granularity of the detected activities was non-trivial. For example, although correctly identified, it was not immediately obvious that the activities _Move to HBW_ and _Hold at HBW_ executed by VGR_1 are indeed two in sequence instead of one single activity.
* Finding similar activities visually based on the same signatures in the IoT data was easy to achieve.
* Distinguishing activities with very similar signatures from each other was only possible by taking the process context (i.e., preceding and succeeding activities) into account. This was the case, for example, for activities _Pickup from DPS and transport to OV_ and _Pickup from OV and transport to DPS_.
* The detection of activities contributes to incrementally developing process knowledge in the analyst's mental model, which in turn facilitates further detection and  
Figure 21: Evaluation IoT data set labeled by the analyst (green), BPM system (red) or both (black).  
disambiguation of other activities from the data set. For example, it was possible to distinguish between _Pickup from DPS and transport to OV_ and _Pickup from OV and transport to DPS_ because in a previous iteration of the method loops, the _Burn_ activity was detected, signaling that OV_1 was active in the time frame between these two activities and the workpiece must have been transported to the oven.
* The execution of a process activity may not manifest itself directly in the IoT data. For example, the process activity _Read Color_ only retrieves the current value of the color sensor. Thus, there may not be an explicit change in the sensor data visible and the execution not be detectable.
* Based on the identified repeated sequences of activity executions and underlying assumptions (cf. Section 4.1), the analyst was able to recreate (discover) the process models for two different instances of the process (cf. Section 2.2.2).