## 4 Evaluation
### Experimental Results and Discussion  
We employ three common metrics to characterize the ability to identify benign and malicious domains. The first is accuracy, which is defined as the ratio of the number of correctly predicted benign and malicious domains to the total number of benign and malicious domains; the second is recall, which is defined as the ratio of the number of correctly predicted malicious domains to the total number of actual malicious domains; and the third is precision, which is defined as the ratio of the number of correctly predicted malicious domains to the total number of predicted malicious domains. The experimental results are presented in Table 5, where each value is the mean of the five-fold cross-validation results. The proposed approach achieves an accuracy of 0.9989, a recall of 0.9977, and a precision of 0.9869, indicating that our approach provides higher accuracy than that of the two previous implementations. Possible explanations for the performance deterioration in the two implementations include the following. First, there are natural limitations to how well benign and malicious domains can be identified based on character patterns alone. In the implementation based on [5], the tendency to identify domain names consisting of eight or more letters as malicious was particularly prominent, and the results caused more than 15,000 false positives for benign domains \(X_{0}\). Next, in the implementation based on [36], uniformly quantifying the importance of words in the domain strings resulted in a large number of misidentified domains. Specifically, the cases of benign domains \(X_{0}\) and Pizd domains \(X_{4}\) were significant, exceeding 100,000 and 23,000, respectively. While Pizd misidentifications might be improved by optimizing the threshold level, false positives for benign domains will also increase. The ability of our proposed approach to navigate these challenges can be attributed to the consideration of differences between words constituting benign and malicious domain names and to its focus on words playing central roles in word graphs. The numbers of misidentified domains are presented in Table 6, where each value is the sum of the five-fold cross-validation results. Our approach achieves an extremely strong identification performance for Banjori \(X_{1}\) and Sisron \(X_{6}\), whereas the other datasets, including the benign domain names, result in some misidentification. Notably, Banjori \(X_{1}\) and Sisron \(X_{6}\) generate domain names by combining random characters with specific words, and the other types generate domain names by concatenating words contained in their own dictionary. Most of these misidentified domain names can be classified into four categories: (a) Domain names consisting of a single word, (b) domain names consisting of words that appear with extremely low frequency in the training datasets, (c) domain names containing some words common to both benign and malicious domains, and (d) domain names comprising words not included in the dictionary. The proposed approach focuses on the relationships among words in domain names and leverages supervised ML to identify benign and malicious domains. However, the properties of this approach naturally make accurate identification difficult in cases (a) and (b). An investigation into the character strings arising in case (c) revealed numerous occurrences of words such as domain, network, host, and local, which are common to both benign and malicious domains. Therefore, it may be possible to improve the performance in this area by excluding words that commonly appear in domain names by following the technique of stop words in natural language processing [49]. The poor performance in case (d) arose from domain names being segmented into meaningless short character strings. Examples of this problem include domain names composed of random character strings,