# Emotion Recognition in Usability Testing: A Framework for Improving Web Application UI Design
## 3 Results
Deep learning models typically process multiple images as a single batch, expecting each image to have a specific format (width, height, and color channels); therefore, the reshape(\(-1\), img_size, img_size, 3) method is used, providing the structure of the training data to the model, including the total number of images and the format of each individual image. Deep learning models often benefit from normalization of image pixel intensities. In this case, the image intensities are normalized from the range [0, 255] to [0, 1] by dividing each pixel value by 255. This improves training efficiency and standardizes the input. The model was trained several times with different values of a set size, epochs, and number of training data depending on the performance of the equipment used to obtain the best result. In order to train the model for more epochs, it was necessary to reduce the number of images used for training and the size of the set, because the equipment used (Lenovo Legion Y540-15 IRH 15.6\" (Lenovo, Beijing, China)/FHD IPS/i5-9300H/RAM 16 GB/SSD 256 GB/Nvidia GeForce GTX1660 Ti/6 GB G6/Windows 10 Home) was not able to handle such a complex task. The test results are shown in Table 1.  
The first verification test of all four trained models was performed by using AI-generated images containing faces with different emotions. The first three models had difficulty recognizing disgust, sadness, and fear because these emotions have certain facial features in common with other emotions: for example, a frightened person may have a gaping mouth as in the case of surprise, or a disgusted face may have furrowed eyebrows to signal an emotion of anger. The fourth model, with the highest accuracy, showed a significant difference from the previous models, successfully recognizing 7 out of 7 emotions (Figure 6). Figure 6: _Cont._  
Since the fourth model showed high accuracy in consistently recognizing seven out of seven emotions using AI-generated images that were not a part of the dataset used to train the model in the first test, a real-time test was conducted to learn how the model would perform in real-world conditions. The second test was conducted with the following setup: (1) the test subject (author--white male, 31 years old) was facing front of the laptop camera; (2) the laptop camera was 720p and fixed focus was used; (3) there were good lighting conditions on the face using natural light sources. The Haar classifier from OpenCV was used for face detection, and the face region was extracted for the detected face. This region was reduced to 224 \(\times\) 224 pixels to prepare the input image for the model. The model then predicted the emotion based on the pre-processed image, and the result is displayed in real time. All seven emotions were recognized during the real-time test, as shown in Figure 7.  
Figure 6: Test results of the fourth trial of the emotion recognition model, recognizing seven out of seven emotions (left side—the image that was used; right side—predicted emotion probabilities: (**a**) Angry; (**b**) Disgusted; (**c**) Fearful; (**d**) Happy; (**e**) Neutral; (**f**) Sad; (**g**) Surprised). Figure 7: _Cont_. Since the real-time tests with the trained emotion recognition model showed promising results, the following software was selected to track the user's actions in the web application. For this purpose, the Hotjar tool was used, which allows monitoring the behavior of website visitors, analyzing browsing habits, and obtaining various information about how users use websites. Hotjar offers several features, but the most important one for this research is the recording feature. The recordings section allows us to monitor user sessions and view recordings that show how users move and interact with pages, and where they encounter problems or challenges. In addition, Google Analytics GA4 software was chosen to collect data on the usability of the web application interface.