# Improved Detection and Response via Optimized Alerts: Usability Study
## 3 Results
### Mixed ANOVA--Intention to Use (IU)  
Two-way mixed ANOVA (mixed ANOVA) with Bonferroni correction, computed using \(\alpha\) = 0.0125, was performed for IU in isolation. \(\alpha\) = 0.0125 was applicable as one quarter of \(\alpha\) = 0.05 given that the TAM measures related to IU represent one of four tests of related measures.  
Mixed ANOVA was again applied, where the within-subjects variables equating to score totals for each of the three study scenarios were represented by _Intention2 Use (IUS1_tot, IUS2_tot, and IUS3_tot)_, and between-subjects factors were again represented by Maximum Visual (Vis_max), labeled as Yes (\(n\) = 59) and No (\(n\) = 22).  
Participants were presented with three scenarios exhibiting security alert output for the results of applied models, where the output was both VAO and TAO. A mixed ANOVA computed using \(\alpha\) = 0.0125 with a Greenhouse-Geisser correction showed that scores varied significantly across scenarios specific to Intention to Use (_IUS1_tot, IUS2_tot,_ and _IUS3_tot_) in tests of within-subject effects, and significantly again when differentiated for Maximum Visual:  
**Scenarios:** (\(F\) (1.447, 114.327) = 24.493, \(p\) < 0.001, np2 = 0.237)  
**Scenarios\(*\)Vis_max:** (\(F\) (1.447, 114.327) = 5.728, \(p\) = 0.009, np2 = 0.068)  
Post hoc tests using the Bonferroni correction revealed that favorable scores for IU decreased insignificantly from Scenario 1 to Scenario 2 by an average of 0.304 points (\(p\) = 0.758) but declined significantly from Scenario 1 to Scenario 3 by an average of 2.692 points (\(p\) < 0.001). A significant decrease was noted from Scenario 2 to Scenario 3 by an additional 2.388 points (\(p\) < 0.001). The differences in scores were not meaningful between Scenarios 1 and 2 (_IUS1_tot_ and _IUS2_tot_) and Maximum Visual (_Vis_max_) = No, but were quite impactful between Scenarios 2 and 3 (_IUS2_tot_ and _IUS3_tot_) and Maximum Visual (_Vis_max_) = No. As is consistent throughout this analysis, there was a significant difference noted in Scenario 3 (_IUS3_tot_) compared to Scenarios 1 and 2, as well as Maximum Visual = Yes versus Maximum Visual = No. Again, a substantial 19% decrease in mean score for Maximum Visual = No was noted in Scenario 3 as compared to Scenario 2, indicating a significant decrease in IU for participants selecting TAO. As is the case for AU, no change in IU was noted for participants selecting VAO for Scenario 2 as compared to Scenario 1. Also noteworthy was the largest percentage of decrease in mean scores of all  
Figure 8: Estimated marginal means—attitude toward using (AU).  
results recorded, specifically for Scenario 3, indicating that intention to use was low for any aspect of Scenario 3, TAO, or VAO.  
Via estimated marginal means between-subjects, where Maximum Visual = Yes or Maximum Visual = No, inclusive only of IU data with \(\alpha\) = 0.0125 and Bonferroni correction, pairwise comparisons yielded a small 1.423 point mean difference in favor of VAO, insignificant at \(p\) = 0.040. As such, there was not a significant main effect of Maximum Visual scores (\(F\) (1, 79) = 4.378, \(p\) = 0.040, \(\eta\)p2 = 0.053) on the level of acceptance of alert output, as indicated by the sum of participants' scores for IU. These results are represented visually in Figure 9.