# Improved Anomaly Detection by Using the Attention-Based Isolation Forest
## 2 Related Work  
**Attention mechanism. ** An attention mechanism can be viewed as an effective method for improving the performance of a large variety of machine learning models. Therefore, there are many different types of attention mechanisms depending on their applications and the models in which attention mechanisms are incorporated. The term \"attention\" was introduced by [PERSON] et al. (2016). Since the publication of this paper, a huge number of models based on attention mechanisms have been found in the literature. There are also several types of attention mechanisms ([PERSON] et al., 2017), including soft and hard attention mechanisms ([PERSON] et al., 2017), local and global attention ([PERSON] et al., 2018), self-attention ([PERSON] et al., 2018), multi-head attention ([PERSON] et al., 2018), and hierarchical attention ([PERSON] et al., 2018). It is difficult to consider all papers devoted to attention mechanisms and their applications. Comprehensive surveys ([PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018) have covered a large part of the available models and modifications of attention mechanisms. Most attention models are implemented as parts of neural networks. In order to extend a set of attention models, several random forest models in which attention mechanisms were incorporated were proposed in ([PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018). A gradient boosting machine to which an attention mechanism was added was presented in ([PERSON] et al., 2018). **Anomaly detection with attention. ** A wide set of machine learning tasks include anomaly detection problems. Therefore, many methods and models have been developed to address them ([PERSON] et al., 2016; [PERSON] et al., 2016; [PERSON] et al. [PERSON] et al. [43] improved the efficiency and effectiveness of anomaly detection and localization during inference by using a progressive mask refinement approach that progressively uncovered the normal regions and, finally, located anomalous regions. A novel self-supervised framework for multivariate time-series anomaly detection via a graph attention network was proposed in [44]. It can be seen from the above works that the idea of applying attention in models for solving the anomaly detection problem was successfully implemented. However, attention was used in the form of components of neural networks. There are no forest-based anomaly detection models that use an attention mechanism. **iForest. ** iForest [16; 17] can be viewed as one of the most important and effective methods for solving novelty and anomaly detection problems. Therefore, many modifications of the method have been developed [5] to improve it. A weighted iForest and Siamese gated recurrent unit algorithm architecture that provided a more accurate and efficient method for outlier detection of data was considered in [45]. [PERSON] et al. [46] proposed an extension of iForest called the extended isolation forest, which resolved issues with the assignment of anomaly scores to given data points. A theoretical framework that described the effectiveness of isolation-based approaches from a distributional viewpoint was studied in [47]. [PERSON] et al. [48] presented a generalized isolation forest algorithm that generated trees without any empty branches, which significantly improved the execution times. The k-means-based iForest was developed by [PERSON] et al. [49]. This modification of iForest allowed one to build a search tree based on many branches, in contrast to the two considered in the original method. Another modification called the fuzzy set-based isolation forest was proposed in [50]. A probabilistic generalization of iForest was proposed in [51], which was based on the nonlinear dependence of a segment-cumulated probability on the length of the segment. A robust anomaly detection method called the similarity-measured isolation forest was developed by [PERSON] et al.