# Improved Anomaly Detection by Using the Attention-Based Isolation Forest
According to the original iForest, the isolation measure is estimated as the mean value of the path lengths over all trees in the forest. However, we can replace the averaging of the path lengths with Nadaraya-Watson regression, where the path length of an instance in each tree can be regarded as a prediction in the regression (the valuein terms of the attention mechanism [26]), and weights (the attention weights) depend on the corresponding tree and the instance (the query in terms of the attention mechanism [26]). In other words, the final prediction of the expected path length in accordance with Nadaraya-Watson regression is a weighted sum of the path lengths over all trees. The weights of path lengths have learnable parameters (learnable attention parameters) that can be computed by minimizing a loss function of a specific form. We aim to reduce the optimization problem to a quadratic programming problem or linear programming problem for which there are many algorithms for solving. In order to achieve this aim, [PERSON]'s \(\epsilon\)-contamination model [27] is proposed to be used to compute the learnable attention parameters. The contamination model allows us to represent attention weights in the form of a linear combination of the softmax operation and learnable parameters with the contamination parameter \(\epsilon\), which can be viewed as probabilities. As a result, the loss function for computing learnable parameters is linear with linear constraints on the parameters as probabilities. After adding the \(L_{2}\) regularization term, the optimization problem for computing attention weights becomes a quadratic one. Our contributions can be summarized as follows:  
1. A new modification of iForest called the attention-based isolation forest (ABIForest), which incorporates an attention mechanism in the form of the Nadaraya-Watson regression to improve the solution of the anomaly detection problem, is proposed. 2. The algorithm of computing attention weights is reduced to solving linear or quadratic programming problems due to the application of [PERSON]'s \(\epsilon\)-contamination model. Moreover, we propose the use of the hinge-loss function to simplify the optimization problem. The contamination parameter \(\epsilon\) is regarded as a tuning hyperparameter. 3. Numerical experiments with synthetic and real datasets were performed to study ABIForest. They demonstrated outstanding results for most datasets. The code of the proposed algorithms can be found at [[https://github.com/[PERSON]/Attention-based-isolation-forest](https://github.com/[PERSON]/Attention-based-isolation-forest)]([https://github.com/[PERSON]/Attention-based-isolation-forest](https://github.com/[PERSON]/Attention-based-isolation-forest)) (accessed on 1 November 2022). This paper is organized as follows. Related work can be found in Section 2. Brief introductions to the attention mechanism, Nadaraya-Watson regression, and iForest are given in Section 3. The proposed ABIForest model is considered in Section 4. Numerical experiments with synthetic and real datasets that illustrate the peculiarities of ABIForest and its comparison with iForest are provided in Section 5. Concluding remarks discussing the advantages and disadvantages of ABIForest can be found in Section 6.