# Improved Anomaly Detection by Using the Attention-Based Isolation Forest
## 5 Numerical Experiments
### Synthetic Datasets  
The first synthetic dataset used for the numerical experiments was the Circle dataset. Its points were divided into two parts concentrated around small and large circles, as shown in Figure 2, where the training and testing sets are depicted in the left and right pictures, respectively. In order to optimize the model parameters in the numerical experiments, we performed cross-validation. Gaussian noise with a standard deviation of 0.1 was added to the data for all experiments. The second synthetic dataset (the Normal dataset) contained points generated from normal distributions with two expectations: \((-2,-2)\) and \((2,2)\). Anomalies were generated \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & \(n_{norm}\) & \(n_{anom}\) & \(d\) \\ \hline Circle (synthetic) & 1000 & 200 & 2 \\ \hline Normal dataset (synthetic) & 1000 & 50 & 2 \\ \hline Credit & 1500 & 400 & 30 \\ \hline Ionosphere & 225 & 126 & 33 \\ \hline Arrhythmia & 386 & 66 & 18 \\ \hline Mulcross & 1800 & 400 & 4 \\ \hline Http & 500 & 50 & 3 \\ \hline Pima & 500 & 268 & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A brief introduction to the datasets. Figure 2: Points from the Circle dataset. from a uniform distribution in the interval \([-1,1]\). The training and testing sets are depicted in Figure 3.  
First, we studied the Circle dataset. The F1-score measures obtained for ABIForest are shown in Table 2, where the F1-score is presented as a function of the hyperparameters \(\epsilon\) and \(\tau\) with the number of trees in the isolation forest of \(T=150\). It is interesting to note that ABIForest was sensitive to changes in \(\tau\), whereas \(\epsilon\) did not significantly impact the results. For comparison purposes, the F1-score measures of the original iForest as a function of the number of trees \(T\) and the hyperparameter \(\tau\) are shown in Table 3. It can be seen from Table 3 that the largest value of the F1-score was achieved with 150 trees in the forest and with \(\tau=0.5\). One can also see from Tables 2 and 3 that ABIForest provided results that outperformed the same results of the original iForest. Similar numerical experiments with the Normal dataset are presented in Tables 4 and 5. We can see again that ABIForest outperformed iForest, namely, the best value of the F1-score \begin{table}
\begin{tabular}{l c c c} \hline \(\epsilon\) & \multicolumn{3}{c}{\(\tau\)} \\ \hline  & 0.5 & 0.6 & 0.7 \\ \hline
0.0 & 0.276 & 0.973 & 0.236 \\ \hline
0.25 & 0.2749 & 0.975 & 0.162 \\ \hline
0.5 & 0.273 & **0.978** & 0.100 \\ \hline
0.75 & 0.273 & 0.975 & 0.062 \\ \hline
1.0 & 0.271 & 0.973 & 0.037 \\ \hline \end{tabular}
\end{table} 
Table 2: F1-scores of ABIForest consisting of \(T=150\) trees as a function of the hyperparameters \(\tau\) and \(\epsilon\) for the Circle dataset with \(\omega=20\). Figure 3: Points from the Normal dataset. \begin{table}
\begin{tabular}{l c c c c c} \hline \(\tau\) & \multicolumn{3}{c}{\(T\)} \\ \hline  & 5 & 15 & 25 & 50 & 150 \\ \hline
0.3 & 0.270 & 0.270 & 0.270 & 0.270 \\ \hline
0.4 & 0.286 & 0.273 & 0.271 & 0.270 & 0.270 \\ \hline
0.5 & 0.729 & 0.864 & 0.899 & 0.906 & **0.920** \\ \hline
0.6 & 0.639 & 0.603 & 0.598 & 0.603 & 0.606 \\ \hline \end{tabular}
\end{table} 
Table 3: F1-scores of the original iForest as a function of the number of trees \(T\) and the hyperparameter \(\tau\) for the Circle dataset. provided by iForest was 0.252, whereas the best value of the F1-score for ABIForest was 0.413, and this result was obtained with \(\omega=20\). Figure 4 illustrates how the F1-score depended on the hyperparameter \(\tau\) for the Circle dataset. The corresponding functions are depicted for different contamination parameters \(\epsilon\), and they were obtained for the case of \(T=150\) trees in iForest. It can be seen from Figure 4 that the largest value of the F1-score was achieved with \(\omega=20\) and \(\epsilon=0.5\).