## 4 Results
### Analysis of Evaluation Metrics
#### 4.2.2 Chatbot Engagement Assessment Using the UES-SF  
The initial segment of the analysis concentrated on comprehending the independent variables of the interaction mode and question complexity influenced the dependent variable of user engagement. The UES-SF was used to assess four dimensions of engagement: Focused Attention (FA), Perceived Usability (PU), Aesthetic Appeal (AE), and reward (RW). Table 6 presents a descriptive analysis of the UES-SF. The hybrid-based interaction mode achieved the highest engagement for simple tasks, with a mean score of 75.2 \(\pm\) 18.91 (median: 79.1, range: 51.6-90.9), outperforming the text-based and menu-based modes. The text-based mode followed closely, with a mean of 65.3 \(\pm\) 20.22 (median: 66.2, range: 40.0-88.9), while the menu-based mode had a slightly higher mean of 68.0 \(\pm\) 22.65 (median: 70.4, range: 40.9-90.2) but more variability across tasks. In the case of complex tasks, the hybrid-based mode maintained its competitive edge, achieving a mean score of 71.9 \(\pm\) 19.41 (median: 77.8, range: 44.0-88.2), indicating that it supported effective engagement across tasks of moderate complexity. The text-based mode also performed well, demonstrating steady usability, with a mean of 68.8 \(\pm\) 16.24 (median: 70.2, range: 47.6-87.1). The menu-based mode, on the other hand, showed improved scores, with 71.3 \(\pm\) 10.26 (median: 73.3, range: 58.7-80.0). The menu-based mode led for complex tasks, with the highest mean score of 77.1 \(\pm\) 3.13 (median: 77.1, range: 73.3-80.9), showing that it could handle high engagement when the task difficulty increased. However, the text-based mode followed closely, with 73.6 \(\pm\) 7.53 (median: 74.9, range: 63.3-81.3), and the hybrid-based mode lagged slightly behind at 73.2 \(\pm\) 14.24 (median: 74.1, range: 54.9-89.6). The variability in the hybrid-based mode's scores for complex tasks indicates that it may struggle to maintain consistent engagement in more demanding scenarios. Overall, the hybrid mode led to superior engagement outcomes across various complexities. Statistically, a 3 \(\times\) 3 repeated-measures ANOVA revealed that the hybrid-based interaction mode significantly outperformed the text-based and menu-based modes in terms of overall engagement (F (2,58) = 8.67, \(p\) < 0.001, \(\eta^{\rm 2P}\) = 0.23). However, increasing question complexity significantly decreased engagement across all modes (F (2,58) = 126.47, \(p\) < 0.001, \(\eta^{\rm 2P}\) = 0.81). \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Interaction Mode** & **Difficulty** & **Mean \(\pm\) SD** & **(Median) Min-Max** \\ \hline \multirow{4}{*}{Text} & Simple & 65.3 \(\pm\) 20.22 & (66.2) 40.0–88.9 \\  & Intermediate & 68.8 \(\pm\) 16.24 & (70.2) 47.6–87.1 \\  & Complex & 73.6 \(\pm\) 7.53 & (74.9) 63.3–81.3 \\  & Simple & 68.0 \(\pm\) 22.65 & (70.4) 40.9–90.2 \\ Menu & Intermediate & 71.3 \(\pm\) 10.26 & (73.3) 58.7–80.0 \\  & Complex & **77.1**\(\pm\) 3.13 & (77.1) 73.3–80.9 \\  & Simple & **75.2**\(\pm\) 18.91 & (79.1) 51.6–90.9 \\ Hybrid & Intermediate & **71.9**\(\pm\) 19.41 & (77.8) 44.0–88.2 \\  & Complex & 73.2 \(\pm\) 14.24 & (74.1) 54.9–89.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Descriptive statistics for UES-SF scores. Regarding reliability, the [PERSON]'s alpha values for the UES-SF engagement dimensions by interaction mode were 0.64, 0.53, and 0.46 for the hybrid-based, menu-based, and text-based modes, respectively, indicating moderate-to-low internal consistency across interaction types. The higher consistency for the hybrid-based mode suggests that users found this mode to be more reliably engaging, likely because of its flexibility in supporting varied engagement levels across task complexities. In contrast, the lower [PERSON]'s alpha values for the menu-based and text-based modes indicate that users experienced more variability in engagement within these modes, as they may not provide the same adaptability or immersive quality offered by the hybrid-based model in educational interactions.