## 4 PER-SAC based Optimal Parameter Autotuning for DBSCAN
### PER-SAC Algorithm
As shown in Figure 10, the results indicate that the parameter space is set too large or too small, which will result in a decrease in the final clustering effect or a decrease in the parameter searching efficiency. Moreover, an appropriate number of recursive layers, i.e., appropriate model size, and appropriate reward factor both help to obtain a stable clustering result and a high parameter searching efficiency. According to the experimental results, the experimental hyperparameters are chosen such that the parameter space size of _Eps_ is set to 5, the parameter space size of _MinPts_ is set to 4, the number of recursive layers is set to 3, and the reward factor is set to 0.2. In addition to the above hyperparameters, the proportion occupied by labels used for model validation is 20%, and the dimensions of FCN (fully connected network) and MLP (multi-layer perceptron) are set to 32 and 256, respectively. The default values of other hyperparameters are shown in Table 6. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Hyperparameters** & **Description** & **Default Values** \\ \hline \(E_{max}\) & Maximum number of rounds of training process & 15 \\ \(B^{0}_{MinPts,2}\) & Maximum search boundary for _MinPts_ in Layer 0 & 0.25 \(\times\) dataset size \\ \(I_{max}\) & Maximum search step in each round of episode & 30 \\ \(\gamma\) & Reward decay factor of Critic networks & 0.1 \\ \(M\) & Number of samples in the prioritized experience & 1 \\  & replay buffer & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Selection of hyperparameters. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithms**} & \multicolumn{2}{c}{**8.5K**} & \multicolumn{2}{c}{**50K**} & \multicolumn{2}{c}{**217K**} \\ \cline{2-7}  & **DBI** & **Silhouette** & **DBI** & **Silhouette** & **DBI** & **Silhouette** \\  & & **Coefficient** & **Coefficient** & **Coefficient** & **Coefficient** \\ \hline DBSCAN [5] & 0.553 & 0.981 & 0.186 & 0.970 & 0.460 & 0.939 \\ RP-DBSCAN [10] & 0.548 & 0.981 & 0.186 & 0.971 & 0.461 & 0.936 \\ STRP-DBSCAN & 0.552 & 0.982 & 0.187 & 0.971 & 0.461 & 0.934 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Evaluation of clustering results using different algorithms with different data sizes. #### 5.3.2 Comparison with Other Parameter Tuning Algorithms

We compare the proposed PER-SAC algorithm with K-Means [4], KANN [25], and the state-of-the-art DRL-based TD3 [31] and SAC [33] algorithms to evaluate the clustering accuracy of the optimal parameters obtained by these algorithms on DBSCAN. In the experiments, we use PER-SAC-DBSCAN to refer to the result that applies the optimal parameters obtained by the PER-SAC algorithm-based framework to the proposed STRP-DBSCAN algorithm. We evaluate all of the comparison algorithms by NMI and ARI values

Figure 10: Effect of hyperparameter selection on the PER-SAC-based framework. of clustering results on 3.3K and 8.5K labeled AIS datasets after preprocessing. Since DRL algorithms such as TD3 and SAC are somewhat randomized, we use the average values and standard deviations of NMI and ARI from ten runs of the algorithms to evaluate the clustering accuracy and stability, respectively. Table 7 shows the evaluation results with 3.3K and 3-classfied dataset. As shown in Table 7, PER-SAC-DBSCAN improves the average NMI and ARI by 16.36% and 35.81% compared to K-Means, and by 22.88% and 42.55% compared to KANN. Compared to TD3-DBSCAN, PER-SAC-DBSCAN shows improvements of 3.07% and 4.22% in the average accuracy of NMI and ARI, and 137.50% and 100% in the stability of both metrics, respectively. Compared to SAC-DBSCAN, it improves the average accuracy of NMI and ARI by 0.88% and 1.32%, and improves the stability of both metrics by 50% and 25%, respectively. We also evaluate the algorithms with an 8.5K and 5-classfied AIS dataset. Table 8 shows the evaluation results.