# Industrial Cyber-Physical System Evolution Detection and Alert Generation
## 5 CALENDAR Evaluation: Scalability Test
### Discussion
#### 5.1.3 F3: Size of the Data Model  
In order to see if the ICPS Data Model size affects the execution time needed, we extracted the values of the random test case (see Appendix A and Table A2). As we have concluded in factor F2, in a real scenario, we do not know about the change that is going to happen, and it has also been shown that time is more or less constant in terms of the number of changes that have occurred. In this manner, we contemplate all the cases without focusing on a single change. As it is shown in Figure 8, the larger the ICPS Data Model, the longer the execution time is. For detecting 100% of changes in a 1000 device Data Model, i.e., 1000 changes, CALENDAR needs an average of 754.174 \(\pm\) 736.894 ms in contrast in the 5000 device Data Model detecting 20% of changes, i.e., for the same quantity of changes (1000), the time needed is bigger (7948.296 \(\pm\) 7740.268 ms). For detecting the same quantity of changes, 546% more time is needed, i.e., equivalent to 6.6% of seconds. Looking at the graph, we know that this is not an isolated case; it is something that occurs if we compare different Data Model sizes. That means that the Data Model size, i.e., the input, impacts the output. The larger the size of the Data Model, the longer the time needed to calculate the differences even though the number of changes is the same.  
In order to see the trend that our system has, we calculated the average needed for each ICPS Data Model size (considering random changes results). Analyzing the results, we can observe that they tend in a potential way, which can be represented as follows: \(y=cx^{a}\).  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **50** & **100** & **500** & **1000** & **5000** & **10000** \\ \hline
**\% of ADD** & 41.89 & 37.27 & 49.21 & 56.20 & 63.51 & 46.49 \\ \hline
**\% of MODIFY** & 7.19 & 7.02 & 7.40 & 12.57 & 6.44 & 0.60 \\ \hline
**\% of REMOVE** & 52.86 & 52.00 & 56.70 & 143.96 & 84.63 & 120.71 \\ \hline
**\% of RANDOM** & 40.92 & 12.08 & 7.71 & 15.49 & 134.10 & 8.19 \\ \hline \hline \end{tabular}
\end{table}
Table 6: The difference between the minimum (remove) and maximum (insert) execution times.  
Figure 8: The time need for detection versus ICPS Data Model sizes (the axes are in logarithmic scale).  
Thanks to a linear regression, i.e., a mathematical model used to approximate the relationship of dependence between a dependent variable (time) and the independent variables (quantity of devices), we obtain values \(a=1.28852412\) and \(c=0.08764925519477608\).  
Thus, we obtain the equation \(f(x)=0.08764925519477608x^{1.28852412}\), which is represented in Figure 9 and shows the relation between the ICPS Data Model size and the execution time.  
Henceforth, using this solution with a high quantity of devices could be a problem, since if CALENDAR needs to be used in an ICPS with many devices, it is necessary to provide a solution to the scalability problem found in order to reduce the execution time. If a fast response is needed when the number of devices is high, this module would not be able to give a fast enough response to the user. For example, in a small (50 Data Model size) scenario, 0.016 \(\pm\) 0.0006 s are needed, but in a bigger one (5000 Data Model size), 6.312 \(\pm\) 0.3912 s are needed. Usually, the latency of an industrial monitoring system is about 2000 ms. That is why our system is not profitable enough in real-time big Data Model scenarios.