# Enhancing User Acceptance of an AI Agent's Recommendation in Information-Sharing Environments
## 3 The Empirical Study
### Results  
We considered only the main effects and two-factor interactions, ignoring higher-level interactions, which we assume to be non-significant, in accordance with the sparsity-of-effects principle. The requirements are defined as a resolution IV experiment. In resolution IV layouts, no main effect is confounded with any other main effect or with any two-factor interaction, but two-factor interactions may be confounded with each other. This design enables the estimation of both main effects that are unconfounded by two-factor interactions and two-factor interactions that may be confounded with other two-factor interactions. Using MATLAB software, we designed the experiment with the E = fracfactgen(a b c d e f g h', [], 5) command and achieved the final design matrix by activating fracfact(E). The final design included 64 experiments, as shown in Table 2. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Experiment** \\ **Number** \\ \end{tabular} } & \multicolumn{8}{c}{**Factors**} \\ \cline{2-9}  & **a** & **b** & **c** & **d** & **e** & **f** & **g** & **h** \\ \hline
1 & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(1\) \\
2 & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(1\) & \(-1\) \\
3 & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(1\) & \(-1\) \\
4 & \(-1\) & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(1\) & \(-1\) & \(1\) \\
5 & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(1\) & \(-1\) & \(-1\) \\
6 & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(1\) & \(-1\) & \(1\) \\
7 & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(1\) & \(-1\) & \(-1\) & \(1\) \\
8 & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(1\) & \(1\) & \(1\) & \(-1\) \\
9 & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(-1\) & \(-1\) & \(1\) & \(1\) \\
10 & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(-1\) & \(1\) & \(-1\) & \(-1\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The empirical study experiment design. Figure 2: Examples of various UI elements. For each factor in each experiment, \(-1\) indicates that this UI element was not applied, and \(1\) indicates that it was applied. The acceptance rate had an average of \(\mu=0.5\) and a standard deviation of \(\sigma=0.32\). About 16% of the participants had an acceptance rate of zero, i.e., did not accept any of the AI agent's recommendations, and about 9% had an acceptance rate of one, i.e., accepted all of the recommendations. The distribution of the acceptance rate is depicted in Figure 3. The \(X\)-axis presents the ranges of acceptance, and the \(Y\)-axis the frequencies. The average acceptance rate for each UI factor separately, i.e., the acceptance rate overall when this specific UI factor is used, is depicted in Figure 4. The \(X\)-axis shows the various UI factors, while the \(Y\)-axis displays the average acceptance rate. The final results were analyzed by using ANOVA (applied with SPSS software v. 26). We first ran the model with all main effects and two-factor interaction combinations, then focused only on the factors and interactions that yielded significant results. The final results are depicted in Table 3.  
Figure 4: The acceptance rate for each factor separately. Figure 3: The distribution of the acceptance rate in the empirical study. It can be seen that the visualization design is the only significant main effect (\(F\)(1,27) = 12.87, \(p\) < 0.01). This is also supported by the effect size calculation, as depicted in Table 4, for the main effects. Motivation was close to significant, but was rejected. Some two-factor interactions may also be considered, e.g., agent provider * performance feedback, but none involved significant main effects. In this specific example, the optimization process is simple and straightforward since there is no need for a combination. The difference between the acceptance rate with or without this UI element is 0.34. The benefit of using the AI agent can be multiplied by 0.34, and the cost of applying this UI element is subtracted from the