# Improved Detection and Response via Optimized Alerts: Usability Study
## 3 Results
### Mixed ANOVA--Perceived Ease of Use (PEU)  
Two-way mixed ANOVA with Bonferroni correction, computed using \(\alpha\) = 0.0125, was performed for PEU in isolation. \(\alpha\) = 0.0125 was applicable as one quarter of \(\alpha\) = 0.05 given that the TAM components related to PEU represent one of four tests of related measures.  
Mixed ANOVA was again applied, where the within-subjects variables equating to score totals for each of the three study scenarios are represented by _Perceived_EaseOfUse (PEUS1_tot, PEUS2_tot, and PEUS3_tot)_, and between-subjects factors were again represented by Maximum Visual (Vis_max), labeled as Yes (\(n\) = 59) and No (\(n\) = 22).  
Participants were presented with three scenarios exhibiting security alert output for the results of applied models, where the output was both VAO and TAO. A mixed ANOVA computed using \(\alpha\) = 0.0125 with a Greenhouse-Geisser correction showed that scores varied significantly across scenarios specific to perceived ease of use (_PEUS1_tot, PEUS2_tot_, and _PEUS3_tot_) in tests of within-subject effects, and insignificantly when differentiated for Maximum Visual:  
**Scenarios:** (\(F\left(1.658,\,130.988\right)\) = 8.752, \(p\) = 0.001, \(\eta\)p2 = 0.100)  
**Scenarios+Vis_max:** (\(F\left(1.658,\,130.988\right)\) = 3.548, \(p\) = 0.040, \(\eta\)p2 = 0.043)  
Post hoc tests using the Bonferroni correction revealed that favorable scores for PEU decreased insignificantly from Scenario 1 to Scenario 2 by an average of 1.020 points (\(p\) = 0.294) but declined significantly from Scenario 1 to Scenario 3 by an average of 3.357 points (\(p\) = 0.002). An insignificant decrease was noted from Scenario 2 to Scenario 3 by an additional 2.337 points (\(p\) = 0.033). The differences in scores were meaningful between Scenarios 1 and 2 (_PEUS1_tot_ and _PEUS2_tot_) and Maximum Visual (_Vis_max_) = No and again between Scenarios 2 and 3 (_PEUS2_tot_ and _PEUS3_tot_) and Maximum Visual (_Vis_max_) = No. A significant difference was, however, noted in Scenario 3 (_PEUS3_tot_) compared to Scenarios 1 and 2, as well as Maximum Visual = Yes versus Maximum Visual = No. Again, a 10% decrease in mean score for Maximum Visual = No was noted in Scenario 3 as compared to Scenario 2, indicating a significant decrease in PEU for participants selecting TAO. Interestingly, there was a 1% increase in PEU for participants selecting VAO for Scenario 2  
Figure 6: Estimated marginal means—PU.  
as compared to Scenario 1. Additionally, for the first time in this analysis, within Scenario 1, TAO outscored VAO within a specific TAM component (PEU).  
Via estimated marginal means between-subjects, where Maximum Visual = Yes or Maximum Visual = No, inclusive only of PEU data with \(\alpha\) = 0.0125 and Bonferroni correction, pairwise comparisons yielded only a 1.229 point mean difference in favor of VAO, insignificant at \(p\) = 0.362. As such, there was not a significant main effect of Maximum Visual scores (\(F\) (1, 79) = 0.842, \(p\) = 0.362, np2 = 0.011) on the level of acceptance of alert output, as indicated by the sum of participants' scores for PEU. These results are best represented visually, as noted in Figure 7.