# Improved Detection and Response via Optimized Alerts: Usability Study
## 3 Results
### Summary of Hypothesis Testing  
The null hypothesis states that there is no statistically significant difference in the level of acceptance of alert output between those choosing VAO and those having some or complete preference for TAO, with VAO and TAO being generated via data science/machine learning methods as predicted by the TAM. The null hypothesis was rejected via non-parametric and parametric methods. Table 9 represents non-parametric outcomes per an independent samples Mann-Whitney U test. The Mann-Whitney U test indicates that there was a significant difference (\(U\) = 863.5, \(p\) = 0.023) between the respondents who selected visual output across all scenarios (\(n\) = 59) as compared to the respondents who provided mixed responses (\(n\) = 22). Table 10 represents the outcomes for parametric tests of within-subjects effects. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Maximum Visual** & & **Mean** & **Std. Deviation** & **N** \\ \hline \multirow{3}{*}{IUS1\_tot} & No & 18.6364 & 2.05971 & 22 \\  & Yes & 18.9661 & 2.66501 & 59 \\  & Total & 18.8765 & 2.50690 & 81 \\ \hline \multirow{3}{*}{IUS2\_tot} & No & 18.0455 & 2.60909 & 22 \\  & Yes & 18.9492 & 2.80039 & 59 \\  & Total & 18.7037 & 2.76335 & 81 \\ \hline \multirow{3}{*}{IUS3\_tot} & No & 14.5909 & 5.11449 & 22 \\  & Yes & 17.6271 & 4.16843 & 59 \\ \cline{1-1}  & Total & 16.8025 & 4.61633 & 81 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Means’ analysis—intention to use (IU). Figure 9: Estimated marginal means—intention to use (IU). The mixed ANOVA using \(\alpha\) = 0.05 with Bonferroni adjustment was significant: (\(F\) (1, 79) = 4.111, \(p\) = 0.046. In summary, the null hypothesis was rejected, as follows:  
* **Non-parametric:**\(U\) = 863.5, \(p\) = 0.023
* **Parametric:**\(\odot\) **Within-subjects:**\((F\) (1.455, 114.915) = 5.634, \(p\) = 0.010, \(\eta\)p2 = 0.067)
* **Between-subjects:**\((F\) (1, 79) = 4.111, \(p\) = 0.046, \(\eta\)p2 = 0.049)  
As such, for RQI: is there a difference in the level of acceptance of security alert output between those with a preference for VAO and those with a preference for TAO, with VAO and TAO generated via data science/machine learning methods, as predicted by the TAM? the answer is yes. Additional sub-questions were examined in this analysis. Specifically, the sub-questions are stated as:  
* [noitemsep,topsep=0 pt]
* SQ1: Does the adoption of VAO have a significant impact on the four individual TAM components: PU, PEU, AU, and IU? * SQ2: Does the adoption of TAO have a significant impact on the four individual TAM components: PU, PEU, AU, and IU? Outcomes indicate mixed results in answering the sub-questions. Table 12 states the results of within-subjects effects per individual TAM components. The within-subjects findings indicated that PU and PEU were not significantly influenced by the adoption of VAO or TAO, while AU and IU were significantly influenced by the adoption of VAO. Table 13 states the results of between-subjects effects per individual TAM components. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Source** & **df** & \(F\) & **Sig.** & **Partial Eta Squared** & **Observed Power** \\ \hline Scenarios * Vis\_max & Greenhouse-Geisser & 1.455 & 5.634 & 0.010 & 0.067 & 0.763 \\ \hline \(\alpha\) = 0.05. * = The impact of vis\_max on the level of acceptance of output as mediated by Scenarios. & & & & \\ \end{tabular}
\end{table} 
Table 10: Tests of within-subjects effects. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Source** & **df** & **F** & **Sig.** & **Partial Eta Squared** & **Observed Power** \\ \hline Vis\_max & Bonferroni & 1 & 4.111 & 0.046 & 0.049 & 0.517 \\ \hline \(\alpha\) = 0.05. & & & & & \\ \end{tabular}
\end{table} 
Table 11: Tests of between-subjects effects.