## 4 Results
### Analysis of Evaluation Metrics
#### 4.2.1 Chatbot Usability: CUQ  
To explore the effects of interaction modes (text-based, menu-based, and hybrid) and question complexity (simple, intermediate, and complex) on user satisfaction (usability), the CUQ scores were analyzed. Table 5 presents the descriptive statistics for the CUQ scores across interaction modes and question complexities. Table 5 indicates that the text-based interaction mode achieved the highest usability scores for simple and intermediate questions, with mean scores of 79.06 \(\pm\) 9.64 (median: 81.25, range: 53.13-100.00) and 77.71 \(\pm\) 11.63 (median: 78.13, range: 53.13-100.00), respectively. For simple tasks, the hybrid-based mode performed comparably, with a mean score of 77.97 \(\pm\) 8.12 (median: 81.25, range: 54.69-87.50), slightly below that of the text-based mode but higher than that of the menu-based mode (76.41 \(\pm\) 8.52, median: 76.56, range: 54.69-90.63). For intermediate questions, the text-based mode continued to perform best, while the hybrid-based mode showed a slightly lower mean score (74.95 \(\pm\) 10.12, median: 76.56, range: 45.31-90.63) but still outperformed the menu-based mode (59.01 \(\pm\) 11.84, median: 57.81, range: 39.06-79.69). In the complex question category, usability scores dropped across all modes, with the text-based mode leading at 63.85 \(\pm\) 14.09 (median: 64.84, range: 32.81-89.06), followed by the hybrid-based mode at 62.08 \(\pm\) 12.06 (median: 62.50, range: 40.63-84.38), while the menu-based mode had the lowest mean score of 43.39 \(\pm\) 12.46 (median: 41.41, range: 17.19-75.00). Statistically, the hybrid-based mode led to superior usability outcomes, with significantly higher CUQ scores (F (2,58) = 89.32, \(p\) < 0.001, \(\eta^{\text{2P}}\) = 0.75) compared to the other modes. Post hoc [PERSON]'s HSD analysis confirmed that the hybrid- and text-based modes scored significantly higher in usability than the menu-based modes (\(p\) < 0.001). While the hybrid- and text-based modes maintained higher usability levels under increased question complexity, the menu-based mode struggled, particularly for complex interactions. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Interaction Mode** & **Diffulty** & **Mean \(\pm\) SD** & **Median (Min, Max)** \\ \hline \multirow{4}{*}{Text} & Simple & **79.06**\(\pm\) 9.64 & 81.25 (53.13, 100.00) \\ \cline{2-4}  & Intermediate & **77.71**\(\pm\) 11.63 & 78.13 (53.13, 100.00) \\ \cline{2-4}  & Complex & **63.85**\(\pm\) 14.09 & 64.84 (32.81, 89.06) \\ \hline \multirow{4}{*}{Menu} & Simple & 76.41 \(\pm\) 8.52 & 76.56 (54.69, 90.63) \\ \cline{2-4}  & Intermediate & 59.01 \(\pm\) 11.84 & 57.81 (39.06, 79.69) \\ \cline{2-4}  & Complex & 43.39 \(\pm\) 12.46 & 41.41 (17.19, 75.00) \\ \hline \multirow{4}{*}{Hybrid} & Simple & 77.97 \(\pm\) 8.12 & 81.25 (54.69, 87.50) \\ \cline{2-4}  & Intermediate & 74.95 \(\pm\) 10.12 & 76.56 (45.31, 90.63) \\ \cline{1-1} \cline{2-4}  & Complex & 62.08 \(\pm\) 12.06 & 62.50 (40.63, 84.38) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Descriptive statistics for CUQ scores. Therefore, the hybrid-based mode demonstrated competitive performance for simple tasks but was less effective for complex tasks, whereas the text-based mode maintained a slight advantage. The reliability of this usability experiment was measured using Cronbach's alpha [75]. The Cronbach's alpha values for the CUQ dimensions by interaction mode were 0.62 for the hybrid-based mode, 0.77 for the menu-based mode, and 0.71 for the text-based mode, indicating moderate-to-good internal consistency across interaction types. These values suggest that, while all modes maintain acceptable reliability, menu-based interactions show the highest consistency in user responses. Our research further expands the findings on the subscales of the CUQ. Figure 3 shows the performance of scores over the average dimensions of the CUQ subscales: personality, onboarding, efficiency, effectiveness, error handling, and learnability.