## 4 Evaluation
### Experimental Setup  
Figure 5 shows the layout of our campus network, which has two class B address blocks. This network consists of one core network, 135 access networks managed by 30 departments, and three wireless networks. We manage only the core and wireless networks, including the connection points to the access networks. The total number of media access control (MAC) addresses observed at their connection points was approximately 9500. A total of 470 access points are located in the three wireless networks that geographically cover most of our campus. The computers of more than 6000 employees and students, in addition to numerous visitors, routinely connect to the three wireless networks. The total number of computers connected to the three wireless networks is approximately 6500 at any given point in time. Broken down by operating system (OS), 43% of those computers use iOS, 28% use Windows, 22% use Android, and 7% use macOS. As expected, there are no details regarding the 135 access networks outside our management area. The dataset used for the experiments comprises DNS queries observed on three RDNSs in the campus network during the one-month period beginning on 1 August 2018, with a total size of approximately 65 GB.5 Note that these three RDNSs were used on our entire campus network, including the access networks outside our management areas. The results show that 3,021,124 queries, i.e., approximately 15% of all queries, meet both a forward lookup request and an NXDOMAIN message response. Table 2 shows the datasets of the benign and malicious domains used in the experiments. The benign domains \(X_{0}\) were the 3,021,124 queries, i.e., approximately 15% of all queries, collected via the campus network described above. We confirmed the absence of malware-related domains in the benign set via the following procedure. First, suspicious computers were detected by multiple security appliances, after which they were examined for matches with the characteristics reported in [6,31,32], which reported that  
Figure 5: Layout of our campus network. a DGA malware will generate massive numbers of NXDOMAIN responses as a result of name resolution for unfamiliar domains during a short period. Notably, even though this approach cannot guarantee the absence of malicious domains with perfect certainty, the number of malicious domains likely to remain among the set of benign domains, if not zero, was considered to be extremely small and unlikely to have a significant impact on our experimental results. In contrast, the malicious domains of seven DGAs \(X_{i},i\in 1\cdots 7\) were published on the websites [45; 46]. Banjori \(X_{1}\) and Sisron \(X_{6}\) dynamically generated domain names by combining random characters with specifically selected words, while the other types dynamically generated domain names by concatenating words in their own dictionaries. Banjori and Sisron are not strictly categorized as dict-DGA malware, but we included them in the datasets because they use meaningful words as domain components. In our experiments, five-fold cross-validation of the benign and malicious domains was performed using 20% of the data as the testing datasets and the remainder as the training datasets. For comparison with our approach, we implemented two different methods for identifying benign and malicious domains based solely on their character strings, as described in [5; 36]. The first implementation used character-level modeling with LSTM networks applied to the domain strings, whereas the second implementation used the number of words that constitute the domain string and are present in the estimated dictionary as the identification threshold. Table 3 shows the specifications of the computer used for implementation and calculation, and Table 4 shows the calculation time in our approach and the two implementations required to train 2,584,900 benign and malicious domains, which is 80% of the datasets, and to predict 646,224 benign and malicious domains, which is 20% of the datasets.