## 2 Backgrounds
### Research on Chatbot Usability and User Experience
Studies like that of ChatPal, a mental health chatbot, highlighted the chatbot's usability (SUS = 64.3) and user satisfaction, showing a strong positive correlation between these metrics [56]. In the education domain, the myHardware chatbot received a SUS score of 61.0, indicating moderate usability [13], while the Aswaja chatbot for promoting Islamic values among youth scored 64.0 on the SUS, suggesting room for improvement [14]. Additionally, the Atreya chatbot for accessing the ChEMBL database in chemistry received a high SUS score of 80.7, demonstrating excellent usability in an academic research context [64]. The development of new evaluation tools like BUS-15 and BUS-11 for assessing chatbot-specific UX shows \begin{table}
\begin{tabular}{p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt}} \hline \hline
**Study** & **Year** & **Focus** & **Chatbot** & **Domain** & **Metrics, Instruments** & **Results** \\ \hline [14] & 2023 & Usability and user satisfaction & Aswaja & Education & SUS & SUS = 64.0 \\ \hline [65] & 2023 & Usability and user satisfaction & ChatGPT & Text Generating & SUS & SUS = 67.4 \\ \hline [53] & 2023 & Usability & Vira & Health & CUQ & CUQ = 70.2 \\ \hline [62] & 2024 & UX and usability & CAI Bing Chat & Higher education & Perceived usefulness, system reliability, system reliability & Positive scores for usability \\ \hline [58] & 2024 & Usefulness & Flapbot & Digital health & SUS & SUS = 68.0 \\ \hline \hline \end{tabular} Note: BUS, Bot Usability Scale; CUQ, Chatbot Usability Questionnaire; I-PANAS-SF, International Positive and Negative Affect Schedule Short Form; PARADISE, PARADigm for DI dialogue System Evaluation; SUS, System Usability Scale; UMUX-LITE, Usability Metric for User Experience LITE; UEQ, User Experience Questionnaire; USE, Usefulness, Satisfaction, and Ease of Use.  
\end{table} 
Table 1: \(Cont.\)advancements in addressing the nuances of conversational agents, including aspects like conversational efficiency and accessibility [11; 12]. The UEQ instrument has been used in chatbot research to evaluate UX, capturing both pragmatic and hedonic aspects of interaction. For example, in the evaluation of the SERMO chatbot, which applies cognitive behavioral therapy for emotional management, the UEQ highlighted positive user feedback on efficiency, perspicuity, and attractiveness while providing neutral scores for hedonic qualities like stimulation and novelty [55]. Similarly, the Differ chatbot, designed to facilitate student community building, was assessed using the UEQ, which revealed that although the overall UX was slightly above average, there was room for improvement in the areas of novelty and stimulation [60]. Additionally, a study on a COVID-19 telehealth chatbot by [PERSON] et al. used the UEQ to assess various facets of the UX, with the chatbot receiving high ratings for ease of use, user satisfaction, and the likelihood of being recommended to others [52]. These studies demonstrate that the UEQ is a valuable tool for providing a holistic view of UX by evaluating chatbot interactions' functional and emotional characteristics. Existing literature lacks standardized tools for evaluating user satisfaction with chatbots. Therefore, [PERSON] et al. developed a new instrument--the Bot Usability Scale (BUS-15), that evaluates chatbot-related aspects like conversational efficiency and accessibility, which are missing in traditional usability evaluation instruments. Their study demonstrated a strong reliability of the BUS-15 and a sound correlation with the UMUX-LITE instrument, making it a significant advancement for chatbot-specific usability assessment. The correlation was evaluated on a sample of 372 surveys; a strong and significant correlation was observed between the overall BUS-15 scale and UMUX-LITE.