# Timed-SAS: Modeling and Analyzing the Time Behaviors of Self-Adaptive Software under Uncertainty
## 5 Experiment Evaluation
### Experiment Results and Discussion
#### 5.2.1 Data Analysis  
According to the checking results of two teachers, Tables 4 and 5 were created. As shown in Table 4, Sub-group A of the five groups, who used the templates of the Timed-SAS approach, all finished modeling and formal analysis of the specified SAS applications, and the average working time was 60.2 min. As for Sub-group B, who directly used the NPTA model, three of five sub-groups (i.e., Group 1-B, Group 2-B and Group 4-B) finished the specific tasks, and the average finishing time was 155.3 min. The other two sub-groups failed in the specific tasks because of timeout (i.e., Group 3-B) or unqualified work (i.e., Group 5-B).  
Table 5 details the reusability for each template created in the Timed-SAS approach. Notably, all the templates were reused in three applications (i.e., the application of Znn.com,  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**SAS Applications**} & **Modeling and Analysis with** & **Modeling and Analysis with** \\  & **the Timed-SAS Approach** & **the NPTA Model** \\ \hline Znn.com [20] & 63 min (Group 1-A) & 195 min (Group 1-B) \\ Lon893 OPCServer [26] & 52 min (Group 2-A) & 163 min (Group 2-B) \\ Body Sensor Network [25] & 75 min (Group 3-A) & /(Group 3-B) \\  & 75 min (Group 3-A) & (out range of 4 h) \\ Fire Detection System [18] & 45 min (Group 4-A) & 108 min (Group 4-B) \\ SUIS & 66 min (Group 5-A) & /(Group 5-B) \\  & & (105 min, but unqualified) \\ Average time & 60.2 min & 155.3 min \\ \hline \hline \end{tabular}
\end{table}
Table 4: The finishing time and work quality for each case study.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**SAS Applications**} & **MAPE-K** & **Uncertainty** & \(\mathbf{M_{\text{Penid}}}\) & \(\mathbf{A_{\text{Only}}}\) & \(\mathbf{Delay_{\text{Tim}}}\) & \(\mathbf{Delay_{\text{Decont}}}\) & \(\mathbf{T_{\text{Adjunt}}}\) & \(\mathbf{T_{\text{Study}}}\) & \
\begin{tabular}{c} **Sas** \\ **Reachabil-** \\ **ity** \\ \end{tabular} &
\begin{tabular}{c} **Strategy** \\ **Reachabil-** \\ **ity** \\ \end{tabular} \\ \hline Znn.com [20] & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ Lon893 OPCServer [26] & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ Body Sensor Network [25] & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ Fire Detection System [18] & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ SUIS & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ Number of uses & 5/5 & 4/5 & 5/5 & 5/5 & 5 Lon893 OPCServer and SUIS). For the above three applications, the templates were directly reused after modifying parameters. In the application of Fire Detection System, the Uncertainty Environment template was not useful, as the danger of fire was a determined value of 0 or 1. In the application of Fire Detection System and Body Sensor Network, the Self-adaptive Steady Time Analysis template was not useful, as the two systems were not closed loop feedback systems. In addition, the NTA model for Fire Detection System in Figure 14, was an instantiated and simplified version of the templates in Figure 8. From the above application and instantiation, we can conclude that the templates created in Timed-SAS can cover modeling and formal analysis requirements of the five example applications.