# An Interactive Method for Detection of Process Activity Executions from IoT Data +
## 4 Method to Identify Activity Executions
### Then Details-on-Demand: Determine Start and End Patterns and Activity Signatures
#### 4.4.1 Activity Start and End Patterns  
From the examples and further observations in the given data set, we can derive the following general conceptual patterns regarding the changes in sensor and actuator values (with \(x\) and \(y\) being arbitrary values) that may be indications for the start or end of an activity (cf. Table 4). These patterns have shown to be valid more generally in CPS [2; 63]. Note that these patterns only refer to _single_ sensors and actuators. In complex CPS that execute more complex processes, these patterns might be combined to _composite patterns_ based on logical conjunctions, aggregations or temporal dependencies between atomic patterns as described in [63] to detect the start or end of an activity. Additionally, we have observed that periods of inactivity followed by one or multiple sensors or actuators becoming active are good indications for the start of an activity and, vice versa, the change from activity to inactivity might indicate the end of activity execution. This holds at least for CPS components that are not operating at high loads (e.g., the oven, sorting machine, and milling machine in our setup). Other components (e.g., the vacuum gripper robot) that are constantly involved in different activity executions require refinements of activities detected solely based on a switch of sensors/actuators from inactive to active. Here, additional component-specific attributes have to be taken into consideration. Regarding sensor characteristics, changes within discrete or even binary states of sensor and actuator values (cf. Table 1) are more likely to indicate activity boundaries than within continuous sensors. Continuous sensors or actuators require additional domain knowledge and discretization into intervals to derive activity executions. Although in our examples we have classified the environmental sensors as irrelevant, changes in their continuous values following specific patterns (e.g., sudden changes or slow increase/decrease) might \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Device** & **Change Pattern** & **Interpretation** & **Example** \\ \hline Sensor & \(0\to x\) & Start or End & light barrier interrupted \\ \hline Sensor & \(x\to 0\) & Start or End & position switch released \\ \hline Sensor & \(x\to y\) & Domain Knowledge & color changed \\ \hline Actuator & \(0\to x\) & Start & motor started \\ \hline Actuator & \(x\to 0\) & End & compressor off \\ \hline Actuator & \(x\to y\) & Domain Knowledge & position reached \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Change patterns of single sensors and actuators and possible interpretations. Figure 12: First part of IoT data filtered by the VGR_1 component with manual annotations. generally also be _effects_ of the activity execution by a different component (e.g., an increase of the environment temperature as a result of starting to burn a workpiece in the oven). We assume these _indirect patterns_ and cross-component dependencies to be part of the domain knowledge of the analyst. However, here we also see a large potential for automating the detection of sensor/actuator correlations across CPS components as part of future work. While the focus of our analysis is on low-level IoT data coming from individual sensors and actuators, CPS components might also emit higher-level sensor data (e.g., discrete state changes within a machine), which should be prioritized when identifying activity executions. This information is either part of the analyst's domain knowledge or can be provided as part of the sensor/actuator characteristics of a CPS component (e.g., in the form of an ontology [64]). Note that in this work, the patterns and sensor-actuator-activity correlations proposed are just indications helping the process analyst to identify potential activity executions from raw IoT data. It is the main responsibility of the process analyst to make ultimate decisions about the start and end of activities.