# Emotion Recognition in Usability Testing: A Framework for Improving Web Application UI Design
## 1 Introduction  
User experience (UX), as defined by the ISO 9241-210 standard [1], encompasses a user's perceptions and responses that result from the use and/or anticipated use of a system, product, or service. This recently developed field of research still faces challenges in defining the scope of UX in general and the application of experiential qualities in particular [2]. The participation of end users is crucial to make interactions as easy and accessible as possible. Usability testing processes help define elements of this area and barriers to implementation [3]. However, for usability--as defined by ISO 9241-210 standard as an extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context--use testing is mostly performed manually, where users provide feedback using usability evaluation methods such as cognitive steps with a think-aloud protocol and a heuristic evaluation survey [4].  
Effectiveness refers to the accuracy and completeness of achieving goals, while performance considers the resources used to achieve these goals. Moreover, satisfaction reflects the convenience and acceptability of the work system for users and affected individuals [3, 5]. These aspects can be effectively evaluated by using affective computing, which involves systems that detect, interpret, process, and simulate human actions. Emotion recognition and the study of behavior resulting from users' emotional states open up new and promising scenarios for a more immersive user experience [6]. The use of this technology in the usability testing process to automate the detection of critical UI areas is not given enough attention, and user research is an important part of the UI/UX design process because it helps designers understand the needs, goals, and preferences of target users. There are several methods designers use to gather user insights [7]: surveys for broad data collection, interviews for detailed individual insights, and usability testing to observe userinteractions with a product. Combining these methods provides a comprehensive assessment of usability and user needs. Researchers often explore measuring and understanding user experience [2] and improving UX practices using data science and process automation, particularly in agile project activities [3]. [PERSON] et al. proposed a UX evaluation technique [8] that helps identify the causes that lead to a negative user experience. It is crucial to understand end users' needs and problems to enhance system usability.  
The most common applications of emotion recognition technologies are in the development of computer games to enhance the overall experience [9]; in education, when trying to improve the teaching and accessibility of the program, by taking into account the emotional state of the students [10]; in medicine to detect health problems earlier [11]; in advertising companies trying to understand what kind of product the respective market wants; or in customer service by providing the best possible service quality. [PERSON] et al. investigated the automatic adaptation of the user interface depending on the multimodal emotion recognition system using the RGB-D sensor [12]. While their focus was on automation rather than UI testing, their work highlights the potential benefits of automating emotion-driven UI adjustments to enhance user experience.  
This research addresses a gap in the field by investigating UI testing based on UX principles using emotion recognition technology. Automating UI usability testing is relevant for saving time and improving data collection. Emotion recognition data can enhance the identification of disliked interface features, informing interface improvements. The proposed framework prototype aims to facilitate usability testing by incorporating emotion recognition data to guide interface testing and identify areas needing improvement. This unique approach could benefit developers seeking user feedback to enhance web application UI.