## 4 Results
### Analysis of Evaluation Metrics
#### 4.2.3 Objective Performance: Efficiency and Effectiveness  
Effectiveness scores, initially measured on a scale of 1 to 5, were normalized to a range of [0, 100] using a linear transformation to facilitate comparison with other metrics. This normalization ensured consistency across metrics, enabling seamless integration into the composite UX metric. The formula for normalization is as follows, using Equation (7):  
\[\text{Normalized Effectiveness Score}\;=\;\frac{(\text{X}-1)}{(5-1)}\times 100 \tag{7}\]  
where:  
* \(X\) represents the original effectiveness score (1 to 5 scale). * Subtraction of 1 adjusts for the scale's lower bound. * Division by 4 adjusts for the range of the original scale. * Multiplication by 100 converts the score to a percentage. Table 7 presents descriptive statistics for effectiveness, while Table 8 summarizes efficiency (response time in seconds). The participants recorded error scores following the guidelines outlined in Table 4, and response times were automatically extracted from the chatbot log files. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Interaction Mode** & **Difficulty** & **Mean \(\pm\) SD** & **(Median) Min–Max** \\ \hline \multirow{4}{*}{Text} & Simple & 35.3 \(\pm\) 8.60 & (40) 20.0–40.0 \\  & Intermediate & 65.3 \(\pm\) 9.00 & (60) 60.0–80.0 \\  & Complex & **84.7**\(\pm\) 17.17 & (90) 60.0–100.0 \\  & Simple & 24.7 \(\pm\) 8.60 & (20) 20.0–40.0 \\ \multirow{4}{*}{Menu} & Intermediate & 55.3 \(\pm\) 17.17 & (50) 40.0–80.0 \\  & Complex & 44.7 \(\pm\) 17.17 & (50) 20.0–60.0 \\ \multirow{4}{*}{Hybrid} & Simple & 35.3 \(\pm\) 8.60 & (40) 20.0–40.0 \\  & Intermediate & **80.7**\(\pm\) 14.37 & (80) 60.0–100.0 \\ \multirow{4}{*}{Menu} & Complex & 64.7 \(\pm\) 17.17 & (70) 40.0–80.0 \\ \cline{1-1} \cline{2-4}  & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Descriptive statistics for effectiveness. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Interaction Mode** & **Difficulty** & **Mean \(\pm\) SD** & **(Median) Min–Max** \\ \hline \multirow{4}{*}{Text} & Simple & 1.41 \(\pm\) 0.12 & (1.5) 1.2–1.5 \\  & Intermediate & 3.48 \(\pm\) 0.34 & (3.35) 3.2–4.0 \\  & Complex & 4.56 \(\pm\) 1.27 & (5.2) 2.5–5.5 \\ \multirow{4}{*}{Menu} & Simple & 1.24 \(\pm\) 0.32 & (1.1) 1.0–1.8 \\  & Intermediate & 3.10 \(\pm\) 0.58 & (2.9) 2.5–4.0 \\ \multirow{4}{*}{Hybrid} & Complex & 2.62 \(\pm\) 1.10 & (2.75) 1.2–3.8 \\  & Simple & 2.04 \(\pm\) 0.50 & (1.9) 1.5–2.8 \\ \multirow{4}{*}{Hybrid} & Intermediate & 4.19 \(\pm\) 0.25 & (4.2) 3.8–4.5 \\  & Complex & 3.66 \(\pm\) 1.44 & (4.3) 1.4–4.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Descriptive statistics for efficiency. Table 7 indicates that the text-based interaction mode achieved the second-highest effectiveness scores for intermediate and complex tasks, with mean scores of 65.3 \(\pm\) 9.00 (median: 60, range: 60.0-80.0) and the highest 84.7 \(\pm\) 17.17 (median: 90, range: 60.0-100.0), respectively. However, for simple tasks, the hybrid-based mode performed poorly, with a mean score of 35.3 \(\pm\) 8.60 (median: 40, range: 20.0-40.0), matching the text-based mode but higher than that of the menu-based mode (24.7 \(\pm\) 8.60, median: 20, range: 20.0-40.0). For complex tasks, the hybrid-based mode showed the highest effectiveness, with a mean score of 80.7 \(\pm\) 14.37 (median: 80, range: 60.0-100.0), significantly outperforming both the text-based (65.3 \(\pm\) 9.00) and menu-based modes (55.3 \(\pm\) 17.17, median: 50, range: 40.0-80.0). In the complex task category, the effectiveness scores varied significantly, with the text-based mode leading at 84.7 \(\pm\) 17.17 (median: 90, range: 60.0-100.0), followed by the hybrid-based mode at 64.7 \(\pm\) 17.17 (median: 70, range: 40.0-80.0), while the menu-based mode had the lowest mean score of 44.7 \(\pm\) 17.17 (median: 50, range: 20.0-60.0). Overall, the text-based mode consistently achieved high effectiveness, especially in complex tasks, reflecting its robustness in handling more demanding scenarios.