## 6 Experimentation
### Results and Statistical Tests
We have two independent non-parametric samples with different variances (where x1 is the mean for Group A and x2 is the mean for Group B, and s1 is the standard deviation for Group A and s2 is the standard deviation for Group B). Equation (7) is used to calculate the value of the significance statistic, because we have two independent non-parametric samples with different variances. Applying this formula, we found that \(p=1.52\). On this ground, H1 would hold considering a significance level of 0.05, as the \(t\)-value, 1.812 (Table 5), is greater than the result for the \(p\)-value. Considering the 0.10 significance level, however, H1 would not hold because 1.52 is greater than 1.372. Figure 16 shows the areas for which there is and there is not any evidence to reject H0, according to \(p\)-value. \[p=(\mathrm{x1-x2})/(\mathrm{s1-s2}) \tag{7}\]  
Table 5 illustrates the decision values for the study carried out, in such a way that in the light of the data obtained the initial hypothesis must be rejected (groups A and B had equivalent results) and the  
Figure 16: H0 rejection area according to \(p\)-value. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Level of Significance** & \(t\)**-Values** & **Probability of H0 Rejection** \\ \hline
0.1 & 1.372 & 1.52 \\
**0.05** & **1.812** & **1.52** \\
0.025 & 2.228 & 1.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of significance level, \(p\)-value with unchanging probability of rejection. hypothesis that the users who used GESTAC spent significantly less time must be accepted. Figure 17 compares the statistical values resulting from applying Equation (7) with the \(p\)-value and level of significance according to the Student's \(t\)-test illustrated in Table 5, if the value of t was unchanged for 10 tests (source: Minitab [62] with GESTAC data). We counted the positive differences between Groups A (\(\upmu\)t1) and B (\(\upmu\)t2) resolution times (Table 6), taking the differences between resolution times specified in Table 1.  
We did not take into account 0 results. If \(\upmu\)A \(\
eq\)\(\upmu\)B, the null hypothesis (H0) was rejected. The statistical summary and its interpretation are shown in Table 7.  
Sign test/Two-tailed test:  
N+ 8  
Expected value 4.000  
Variance 2.000  
\(p\)-value 0.008  
alpha 0.05  
The \(p\)-value is computed using an exact method. \begin{table}
\begin{tabular}{c c} \hline \hline
**Incidents** & **Sign** \\ \hline
1 & + \\
2 & + \\
3 & + \\
4 & + \\
5 & + \\
6 & + \\
7 & \\
8 & + \\
9 & + \\
10 & + \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Data according to difference sign. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Variable** & **Observations** & **Minimum** & **Maximum** & **Mean** & **Standard Deviation** \\ \hline
120 & 9 & 20.000 & 180.00 & 72.222 & 46.711 \\
30 & 9 & 15.000 & 105.00 & 45.000 & 26.693 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary statistics. Figure 17: Graph of statistical values plotted from Table 5.  
Test interpretation:  
H0: the two samples follow the same distribution  
H1: the distribution of the two samples is different. As the computed \(p\)-value is lower than the \(t\)-value at the significance level alpha = 0.10, the null hypothesis H0 should be rejected, and the alternative hypothesis H1, accepted. The risk of rejecting the null hypothesis H0 when it is true is less than 0.78%. On this ground, we can infer that H1 held and H0 was rejected. To check this, we can apply Equation (8) to the data to gather the value of the test statistic (Z), where x is the maximum number of positive data items and n is the total number of data. Equation (8) is used to calculate the critical value for the sign test. As Z is greater than the critical value shown in Table 5 for 0.05, we can infer that the data are statistically significant. \[Z=(2x-n)/(\sqrt{n}) \tag{8}\]  
The decision rule taken was that if the number of positive signs exceeds the critical number, H0 could be rejected.