# Improved Detection and Response via Optimized Alerts: Usability Study
## 3 Results
### Mixed ANOVA--Attitude toward Using (AU)  
Two-way mixed ANOVA with Bonferroni correction, computed using \(\alpha\) = 0.0125, was performed for AU in isolation. \(\alpha\) = 0.0125 was applicable as one quarter of \(\alpha\) = 0.05 given that the TAM measures related to AU represented one of four tests of related measures.  
Mixed ANOVA was again applied, where the within-subjects variables equating to score totals for each of the three study scenarios were represented by _Attitude2 Use_ (_AUS1_tot_, _AUS2_tot_, and _AUS3_tot_), and between-subjects factors were again represented by Maximum Visual (Vis_max), labeled as Yes (\(n\) = 59) and No (\(n\) = 22).  
Participants were presented with three scenarios exhibiting security alert output for the results of applied models, where the output was both VAO and TAO. A mixed ANOVA computed using \(\alpha\) = 0.0125 with a Greenhouse-Geisser correction showed that scores varied significantly across scenarios specific to attitude toward using (_AUS1_tot_, _AUS2_tot_, and _AUS3_tot_) in tests of within-subject effects, and significantly again when differentiated for Maximum Visual:  
**Scenarios:** (\(F\) (1.669, 131.861) = 20.605, \(p\) < 0.001, np2 = 0.207)  
**Scenarios**Vis_max:** (\(F\) (1.669, 130.988) = 8.159, \(p\) = 0.001, np2 = 0.094)  
Post hoc tests using the Bonferroni correction revealed that favorable scores for AU decreased insignificantly from Scenario 1 to Scenario 2 by an average of 0.196 points (\(p\) = 1.000) but declined significantly from Scenario 1 to Scenario 3 by an average of 2.293 points (\(p\) < 0.001). A significant decrease was noted from Scenario 2 to Scenario 3 by an additional 2.097 points (\(p\) < 0.001). The differences in scores were not meaningful between Scenarios 1 and 2 (_AUS1_tot_ and _AUS2_tot_) and Maximum Visual (_Vis_max_) = No, but were quite impactful between Scenarios 2 and 3 (_AUS2_tot_ and _AUS3_tot_) and Maximum Visual (_Vis_max_) = No. As is consistent throughout this analysis, there was a significant difference noted in Scenario 3 (_AUS3_tot_) compared to Scenarios 1 and 2, as well as Maximum Visual = Yes versus Maximum Visual = No. A stark 19% decrease in mean score for Maximum Visual = No was noted in Scenario 3 as compared to Scenario 2, indicating a significant decrease in AU for participants selecting TAO. No change in AU was noted for participants selecting VAO for Scenario 2 as compared to Scenario 1. Also noteworthy was  
Figure 7: Estimated marginal means—perceived ease use (PEU).  
the lowest mean scores of all results recorded, specifically for TAO in Scenario 3, indicating a particularly poor attitude towards using TAO.  
Via estimated marginal means between-subjects, where Maximum Visual = Yes or Maximum Visual = No, inclusive only of AU data with \(\alpha\) = 0.0125 and Bonferroni correction, pairwise comparisons yielded a small 1.587 point mean difference in favor of VAO, insignificant at \(p\) = 0.036. As such, there was not a significant main effect of Maximum Visual scores (\(F\) (1, 79) = 4.566, \(p\) = 0.036, np2 = 0.055) on the level of acceptance of alert output, as indicated by the sum of participants' scores for AU. These results are best represented visually, as noted in Figure 8.