## 4 Experiments  
### Datasets and Indicators  
This paper builds datasets based on Alexa [51] and UMUDGA [52]. Alexa contains nearly 1,000,000 benign domain names, and UMUDGA was released in 2020 and contains 50 families of AGDs, including 6 families with 11 variants of dictionary-based AGDs and 32 families with 39 variants of character-based AGDs. Domain names in some character-based DGA families are also similar to dictionary-based AGDs, such as Vawtrak_v3, but most of the words do not have English meanings, so they are also regarded as character-based AGDs. In our experiments, we regarded each variant as an independent DGA family. Earlier papers usually used DGArchive [53] and Netlab 360 DGA [54] to construct their datasets, but these AGDs are derived from real network traffic, and the number of samples of each family is quite different in the two datasets, which makes them difficult to use. UMUDGA is a relatively standardized dataset that has been used for experiments in many papers since its release in 2020. Here, we selected 500,000 domain names from Alexa and 500,000 AGDs from UMUDGA (10,000 for each DGA family) to build our datasets. We constructed 3 datasets: a dictionary-based dataset that contained 210,000 samples, a character-based dataset that contained 790,000 samples, and a full dataset that contained 1,000,000 samples. For each dataset, 70% of the samples were used as the training set, 10% as the validation set, and the other 20% as the testing set. The validation set was used to estimate the fitting state of the model during the training stage. When the accuracy of the validation set decreased significantly, the epoch with the highest accuracy on the validation set was selected as the final model, and the performance of the model on the testing set was recorded as the final result. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Type** & **Feature Name** & **Description** & **Whether Need** \\ \hline Domain & Len\_total & Total length of original domain name & Yes \\ \cline{2-3} length features & Len\_no\_td & Total length of domain name & Yes \\ \hline  & Ratio\_vow & Ratio of vowel characters in domain & \\  & Ratio\_conv & name without TLD & No \\ \cline{2-3} Character & Ratio\_con & Ratio of consonant characters in & \\  & ratio & domain name without TLD & No \\ \cline{2-3} features & Ratio\_dig & Ratio of digit characters in domain & No \\ \cline{2-3}  & If\_contain & Whether to contain hyphen (“-”). & No \\  & \_hypen & 1 indicates yes and 0 indicates no & No \\ \hline  & Num\_words & The total number of words after & Yes \\ \cline{2-3}  & & meaningful word segmentation & Yes \\ \cline{2-3} Word & Avg\_word & The average length of words after & \\  & \_len & meaningful word segmentation & Yes \\ \cline{2-3} Word & Ratio\_JJ & The ratio of adjectives in a word after & No \\ \cline{2-3} features & Ratio\_NN & The ratio of nouns in a word after & \\  & meaningful word segmentation & No \\ \cline{2-3}  & Ratio\_VB & The ratio of verbs in a word after & \\  & meaningful word segmentation & No \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of statistical features. In addition, we constructed 3 generalization testing sets based on the AGDs from DGA Archive and Netlab 360 to measure the real-world detection performance of our final model and the three comparison methods, which were trained on the full dataset. First, we constructed 2 generalization testing sets based on the AGDs from DGA Archive. We selected up to 5,000 AGDs from each family, based on whether or not the family was included in the UMUDGA, the AGDs were divided into two generalization testing sets. Then, we selected up to 1,000 AGDs for each family after October 2022 from Netlab 360 and built the third generalization dataset. The details of our final datasets are shown in Table 2.  
For the three datasets, the number of positive samples (AGD) and negative samples (benign) were roughly equal, so the accuracy, precision, recall and F-measure could be used to evaluate the model effects.