# Improved Detection and Response via Optimized Alerts: Usability Study
## 3 Results
### Mixed ANOVA--All Measures (PU, PEU, AU, IU Combined)  
A two-way mixed ANOVA was conducted, with a Bonferroni correction for the within-subjects variable. The dependent variable was the level of acceptance of alert output, with all items of all TAM sub-scales summed. While considered more conservative, most authorities suggest the Greenhouse-Geisser correction when the epsilon (\(\epsilon\)) estimate is below 0.75. As noted in Table 8, \(\epsilon=0.727\), and thus the Greenhouse-Geisser correction was utilized. As indicated in Table 8, sphericity cannot be assumed as \(p<0.001\). As such, the Greenhouse-Geisser correction was applied. The within-subjects variable, equating to score totals for each of the three study scenarios, is represented by Scenarios (_S1_tot_, _S2_tot_, and _S3_tot_). The between-subjects variable was Maximum Visual (Vis_max), labeled as _Yes_ (\(n=59\)) and _No_ (\(n=22\)). Again, the Maximum Visual variable (_Vis_max_) differentiates between the participants who said yes to VAO in all three scenarios, labeled _Yes_ (N = 59), and the participants who selected a mix of VAO and TAO, or all TAO results, across all three scenarios, labeled _No_ (\(n=22\)). Maximum Visual is the statistical analogy for the study's between-subjects independent variable, specifically (a) ML/DS-generated TAO, and (b) ML/DS-generated VAO. Participants were presented with three scenarios exhibiting security alert output for the results of applied models, where the output was both VAO and TAO. A mixed ANOVA using \(\alpha=0.05\) with a Greenhouse-Geisser correction showed that scores varied significantly across Scenarios in tests of within-subject effects, and there was also a significant interaction with Maximum Visual:  
**Scenarios:** (\(F\) (1.455, 114.915) = 19.925, \(p<0.001\), \(\eta\)p2 = 0.201)  
**Scenarios**Vis_max:** (\(F\) (1.455, 114.915) = 5.634, \(p=0.010\), \(\eta\)p2 = 0.067)  
The impact of Maximum Visual (vis_max) on the level of acceptance of output was mediated by Scenarios. The difference of the level of acceptance was more significant for Scenario 3, as an example. Post hoc tests using the Bonferroni correction revealed that favorable scores declined insignificantly from Scenario 1 to Scenario 2 by an average of 1.596 points (\(p=0.702\)) but declined significantly from Scenario 1 to Scenario 3 by 12.342 points (\(p<0.001\)). Scenario 2 to Scenario 3 saw an additional significant decrease of 10.746 points (\(p<0.001\)). The differences in scores were not particularly meaningful between or within Scenarios 1 and 2 (_S1_tot_ and _S2_tot_) and Maximum Visual (_Vis_max_) = Yes or No. However, a significant difference was noted in Scenario 3 (_S3_tot_) compared to Scenarios 1 and 2, as well as Maximum Visual = Yes versus Maximum Visual = No. Most noteworthy is a 15% decrease in mean score for Maximum Visual = No in Scenario 3 as compared \begin{table}
\begin{tabular}{l r} \hline \hline Total N & 81 \\ Test Statistic & 5.496 \\ Degree of Freedom & 2 \\ Asymptotic Sig. (2-sided test) & 0.064 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Related samples [PERSON]’s two-way ANOVA by ranks, summary. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Within-Subjects Effect** & **Mauchly’s W** & \
\begin{tabular}{l} **Approx.** \\ **Chi-Square** \\ \end{tabular} & **df** & **Sig.** &
\begin{tabular}{l} **Greenhouse–** \\ **Geisser** \\ \end{tabular} \\ \hline Scenarios & 0.625 & 36.652 & 2 & 0.000 & 0.727 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: [PERSON]’s test of sphericity. to Scenario 2, indicating a noteworthy decrease in PU, PEU, AU, and IU for participants selecting TAO. Via estimated marginal means between-subjects, where Maximum Visual = Yes or Maximum Visual = No, inclusive of all TAM components with \(\alpha\) = 0.05 and Bonferroni correction, pairwise comparisons yielded a 7.881 point mean difference in favor of VAO, significant at \(p\) = 0.046.