## 2 Methodology
### Classification of GDSW Based on Machine Learning
#### 2.4.1 Classification Algorithm
Similarly, we manually marked 628 websites filtered by multi-source semantic information, and obtained and trained the training set by the same method, but the prediction set was the same as that mentioned above to verify the effectiveness of multi-source semantic information for classification (the second set of experiments). See Section 3.1 for details. To verify the robustness of the proposed method, we obtained the second set of data using different keyword groups based on the Baidu search engine and used the same method for training and evaluation, as described in Section 3.2. ### Evaluation of Classification Results Based on Machine Learning

We performed the training phase using KNN, LR, RF, and SVM for the above training set, and set the main parameters of each classifier as shown in Table 1. After the training phase, we used the learned classifier to predict the categories of all records in the test set and compared them with the real categories to calculate the confusion matrix. Table 2 shows the classification precision, recall rate, and F1-score calculated according to the confusion matrix. It can be seen that, when the pre-processed sample data are used directly for classification, the highest precision can reach up to 80% using RF and SVM. For the same classification precision of utilizing RF and SVM, the recall and F1-score of SVM are higher. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**KNN** & **LR** & **RF** & **SVM** \\ \hline n\_neighbors = 35 & max\_iter = 1100 & n\_estimators = 100 & Kernel = ’linear’ \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Main parameter setting of KNN, LR, RF and SVM. After that, we trained the second group of experimental data using the four classification methods, and predicted the test set in the first group of experimental data. The results show that (as shown in Table 3) the classification precision was greatly improved in all four classification algorithms compared with using the sample data that were only pre-processed. The classification precision (95%) and the corresponding F1-score (58.5%) of SVM are the highest, accordingly. Combined with the precision-recall (PR) curve (as shown in Figure 5), it can be seen that, overall, the classification precision of SVM is also better with consistent recall, and the classification precision of LR is relatively higher only when recall is greater than 0.75. Meanwhile, for the ROC curve (as shown in Figure 6), the classification performance of all four classification algorithms is good, and their AUROC values are all floating around 0.9 with little fluctuation. The AUROC value of LR (0.93) is the highest, followed by SVM (0.92). When the classification performance is good in all cases, we usually select the classifier with higher classification precision because our goal is to precisely identify GDSW. ### Robustness

To fully verify the robustness and generalizability of the proposed method, we selected keywords from the data classification system of the National Earth System Science Data Center ([[http://www.geodata.cn/](http://www.geodata.cn/)]([http://www.geodata.cn/](http://www.geodata.cn/)), last accessed: 15 September 2021), which are different from those keywords selected in the literature [32] and describe the data in finer granularity. According to the method in Section 2, we obtained a total of 81,499 URLs, and obtained 1852 original sample data (called robust data) by domain name extraction and deduplication. Then, we obtained 921 sample data by filtering the 1852 initial sample data with multi-source semantic information. Finally, we applied the method in Section 3.1 to train and evaluate the robustness. As illustrated in Table 4, when we used the data filtered with multi-source semantic information as the sample data, SVM had the highest classification precision and F1-score, which were 84.2% and 78%, respectively. Overall, the classification precision decreased compared with the value in Section 3.1, but the corresponding recall and F1-score were improved.