# Implementing and Evaluating a Font Recommendation System Through Emotion-Based Content-Font Mapping
## 5 Usability Evaluation of the Font Auto-Recommendation System
### Usability Evaluation
A total of 15 participants took part in the evaluation. In the evaluation, the provided font groups were referred to as Group 1, which includes the top 6 recommended fonts; \begin{table}
\begin{tabular}{c c c} \hline \hline
**Evaluation Criteria** & & **Measurement Methods** \\ \hline \multirow{4}{*}{Appropriateness} & \(\bullet\) & Verify the groups in which the content and recommended fonts have been classified according to their rankings. \\  & \(\bullet\) & Select the group of fonts that best matches the content. \\  & \(\bullet\) & Select the group of fonts that least suits the content. \\ \multirow{4}{*}{Satisfaction} & \(\bullet\) & Satisfaction evaluation of recommended fonts by font group (0–6 points). \\  & \(\bullet\) & Selecting fonts suitable for desired content through the existing font recommendation system [13]. \\ \multirow{4}{*}{Efficiency} & \(\bullet\) & Selecting fonts suitable for the desired content through our system. \\  & \(\bullet\) & Evaluating the efficiency of this system compared to the existing system (0–6 points). \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Measurement methods and evaluation scales for usability evaluation. Group 2, which includes fonts ranked 7 th to 12 th; Group 3, which includes fonts ranked 13 th to 19 th; and Group 4, which includes fonts ranked 20 th to 26 th. Based on the assessment of 12 different contents presented to users, Group 1, which contains the fonts recommended as the top-ranked ones, was consistently rated as the most suitable font group for each content. On average, 10.67 participants rated Group 1 as the most appropriate. In contrast, Group 4, which contains fonts with the lowest rankings, received an average rating of 1.68, with participants finding it suitable. Group 3 received the lowest average rating, with 0.94 participants finding it appropriate. However, statistically, there was no significant difference in ratings between Groups 2, 3, and 4. The graph depicting the appropriateness of the evaluation results is shown in Figure 6.  
Conversely, in the evaluation where participants were asked to select the least appropriate font group, Group 4 received the highest number of selections, with approximately 7.9 participants choosing it, while Group 1 was chosen by the fewest participants, with only 0.75 individuals selecting it. However, for Groups 2, 3, and 4, there was no statistically significant difference in the choices made by participants. This can be seen in the following Figure 7.  
When each font group was individually presented to users and font recommendation satisfaction was evaluated on a scale from 0 to 6 using the Likert score, Group 1 received the highest satisfaction with a score of 4.68. It was observed that as we move from Group 1  
Figure 6: The average count of respondents who considered the results appropriate was classified by font recommendation rankings among grouped categories. Figure 7: The average count of respondents who deemed the results inappropriate, categorized by font recommendation rankings among grouped categories. to Group 4, the average satisfaction gradually decreases. The group-specific results for satisfaction scores are illustrated in Figure 8.  
Finally, we aimed to assess the efficiency of our system through a comparison with the existing font search system, as illustrated in Figure 9 [13]. As a result, users highly evaluated the system that automatically recommends fonts based on content input, with an average efficiency score of 5.27. This score was measured after comparing it with the sentiment-based font recommendation system developed in previous research. The methodology involved initially having 15 participants use the existing system, followed by using our system. Participants were asked to rate the efficiency of the new sentiment-based font recommendation system compared to the existing one.