## 4 PER-SAC based Optimal Parameter Autotuning for DBSCAN
### PER-SAC Algorithm
We define the immediate reward function of the \(i\)-th step as Equation (5). \[R(s^{(e)(i)},a^{(e)(i)})=NMI(DBSCAN(X,P^{(e)(i+1)}),y^{\prime}) \tag{5}\]

Here, \(s^{(e)(i)}\) and \(a^{(e)(i)}\) indicate the state and action of the \(i\)-th step of the \(e\)-th episode, respectively. \(NMI\) () is the normalized mutual information (NMI) representing the external metric function of DBSCAN clustering. \(X\) is the feature set and \(y^{\prime}\) is the set of partial labels of the data block. NMI is commonly used to measure the accuracy of the results of clustering algorithms and we use it as the reward function. Furthermore, the optimal parameter search action sequence for an episode is to tune the parameters in the direction of the optimal parameters and stop the search at the optimal parameters. Therefore, we consider using the maximum immediate reward of subsequent steps and the endpoint immediate reward as the reward of \(i\)-th step, see Equation (6). \[r^{(e)(i)}=\beta\cdot\max\Bigl{\{}R(s^{(e)(m)},a^{(e)(m)})\Bigr{\}}|_{m=i}^{I }+\delta\cdot R(s^{(e)(I)},a^{(e)(I)}) \tag{6}\]

Figure 9: Parameter autotuning process of the framework in a single episode. Here, \(R(s^{(e)(l)}\), \(a^{(e)(l)})\) is the immediate reward for the \(I\)-th step end point parameter. And the max function is used to calculate the future maximum immediate reward before stopping the search in the current episode. \(\beta\) and \(\delta\) are the impact factors of reward, where \(\beta\) = \(1-\delta\). The process of parameter search is repeated for each recursive layer in the training process to optimize the agents, and the optimal parameter set of {_Eps_, _MinPts_} is updated based on the reward function. An early stopping mechanism is used to speed up the model training process while the optimal parameter set remains unchanged. When the same parameters are obtained more than three consecutive times, the training is aborted, which in turn speeds up the training process. While in the testing process, the trained agent is directly used to search in the batch, and the early stopping mechanism is no longer used. The parameters of the last layer after executing an episode can be used as the optimal parameter set. Algorithm 4 shows the optimal parameter autotuning process of the framework for DBSCAN. ```
Input: The features of dataset X, partial labels y' of data block V; Agents for each layer: \(\left\{agent^{(l)}\right\}|\{l_{\max}\); Output: Optimal parameter set: \(P_{0}\);
1 for\(l=1,...,l_{\max}\)do
2 //Initialize parameter space
3 Initialize the space boundaries of parameter \(p\) in the \(l\)-th layer \(B_{p,1}^{(l)}\), \(B_{p,2}^{(l)}\), the search precision of the parameter \(p\) in the \(l\)-th layer \(\theta_{p}^{(l)}\);
4 for\(e=1,...,E_{\max}\)do
5 //Initialize parameter set
6 Initialize \(P^{(e)(0)}\) by \(P_{o}^{(l-1)}\);
7 //Parameter autotuning
8 for\(i=1,...,l_{\max}\)do
9 Obtain the current state \(s^{(e)(0)}\);
10 Choose \(a^{(e)(l)}=\)\(\mathit{Actor}(s^{(e)(l)})\);
11 get \(p^{(e)(i)}\)\(a^{(e)(i)(0)}\),\(p^{(e)(i+1)}\);
12 DBSCAN(\(p^{(e)(i)}\));
13 Termination judgment;
14 endfor
15 if is TRAIN then get rewards
16 \(r^{(e)(i)}=\beta\cdot\max\Bigl{\{}R(s^{(e)(m)},a^{(e)(m)})\Bigr{\}}|_{m=i}^{l} +\delta\cdot R(s^{(e)(l)},a^{(e)(l)})\);
17 Store \(\Gamma^{(e)(i)}=(s^{(e)(i)},a^{(e)(i)},s^{(e)(i+1)},r^{(e)(i)})\) in experience replay buffer;
18 Sampling and learning;
19
20 endif
21 //Update optimal parameter set \(P_{o}^{(l)}\)
22 update \(P_{o}^{(l)}\);
23 Early stop judgment;
24 endfor
25 update \(P_{o}\);
26 Early stop judgment;
27 endfor
```

**Algorithm 4**Optimal parameter autotuning process for DBSCAN

Regarding the computational complexity of parameter search, for ease of presentation, we define \(\pi_{p}\) to be the number of searchable parameters in the parameter space of parameter \(p\) in each layer. Additionally, \(\theta_{p}^{(l)}\) is the search step size in the \(l\)-th layer, which can be defined as Equation (7). \[\theta_{p}^{(l)}=\left\{