# Improved Anomaly Detection by Using the Attention-Based Isolation Forest  
[PERSON]  
1 Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University,  
Polytechnicheskaya, 29, 195251 St. Petersburg, Russia  
[PERSON]  
1 #  
2  
[PERSON]  
1 Higher School of Artificial Intelligence, Peter the Great St. Petersburg Polytechnic University,  
Polytechnicheskaya, 29, 195251 St. Petersburg, Russia  
1 #  
2  
[PERSON]  
1 #  
2  
Footnote 1: email: {[PERSON]. [PERSON], [PERSON], [PERSON], [PERSON], A.  
is larger than a predefined threshold, which is usually regarded as a hyperparameter of the model. The next group contains distance-based anomaly detection models. One of the most popular and effective models from this group is the isolation forest (iForest) [16; 17], which is a model for detecting anomalous points relative to a certain data distribution. According to iForest, anomalies are detected by using isolation which measures how far an instance is from the rest of the instances. iForest can be regarded as a tool for implementing isolation. It has linear time complexity and works well with large amounts of data. The core idea behind iForest is the tendency for anomalous instances in a dataset to be more easily separated from the rest of the sample (isolated) compared to normal instances. To isolate a data point, the algorithm recursively creates sample partitions by randomly choosing an attribute and then randomly choosing a split value for the attribute between the minimum and maximum values allowed for that attribute. The recursive partition can be represented by a tree structure called an isolation tree, while the number of partitions needed to isolate a point can be interpreted as the length of the path within the tree to the end node, starting from the root. Anomalous instances are those with a shorter path length in the tree [16; 17]. iForest is a powerful tool for solving the anomaly detection problem. However, it may not take into account that some trees are built in such a way that they do not properly detect anomalous instances because of the random selection of instances for training each tree, the randomized splitting procedure, etc. It is supposed in iForest that all isolation trees behave in a similar way. However, the ranges between the trees' abilities to isolate anomalous instances may be large. Moreover, one anomalous instance can be accurately detected by a tree, whereas another anomalous instance may be undetectable by the same tree. In order to improve iForest, we propose modifying it by using an attention mechanism that can automatically distinguish the relative importance of instances and weigh it to improve the overall accuracy. The attention mechanism allows us to take different capabilities of isolation trees conditioned on instances into account by assigning attention weights to trees and instances depending on their properties. This allows us to compensate for some of the impact of \"bad\" trees, which are \"bad\" only for certain instances and can be accurate for other instances. Attention mechanisms have been successfully applied in many applications, including natural language processing models, the computer vision area, etc. Comprehensive surveys of properties and forms of attention mechanisms and transformers can be found in [18; 19; 20; 21; 22]. The idea of applying an attention mechanism to iForest stems from the attention-based random forest (ABRF) models proposed in [23], where attention was implemented in the form of the Nadaraya-Watson regression [24; 25] by assigning attention weights to the leaves of trees in a specific way such that the weights depended on the trees and instances. The learnable attention parameters in ABRF were trained by solving a standard quadratic optimization problem with linear constraints. It turned out that this idea of considering a random forest as Nadaraya-Watson regression [24; 25] could be extended to iForest by taking the peculiarities of iForest that cause it to differ from the random forest into account.