Package: anomatools
anomatoolsanomatoolsis a small Python package containing recentanomaly detection algorithms.
Anomaly detection strives to detectabnormaloranomalousdata points from a given (large) dataset.
The package contains several state-of-the-art semi-supervised and unsupervised anomaly detection algorithms.InstallationInstall the package directly from PyPi with the following command:pipinstallanomatoolsOR install the package using thesetup.pyfile:pythonsetup.pyinstallOR install it directly from GitHub itself:pipinstallgit+https://github.com/Vincent-Vercruyssen/anomatools.git@masterContents and usageSemi-supervised anomaly detectionGiven a dataset with attributesXand labelsY, indicating whether a data point isnormaloranomalous, semi-supervised anomaly detection algorithms are trained using all the instancesXand some of the labelsY.
Semi-supervised approaches to anomaly detection generally outperform the unsupervised approaches, because they can use the label information to correct the assumptions on which the unsupervised detection process is based.
Theanomatoolspackage implements two recent semi-supervised anomaly detection algorithms:TheSSDO(semi-supervised detection of outliers) algorithm first computes an unsupervised prior anomaly score and then corrects this score with the known label information [1].TheSSkNNO(semi-supervised k-nearest neighbor anomaly detection) algorithm is a combination of the well-knownkNNclassifier and thekNNO(k-nearest neighbor outlier detection) method [2].Given a training datasetX_trainwith labelsY_train, and a test datasetX_test, the algorithms are applied as follows:fromanomatools.modelsimportSSkNNO,SSDO# traindetector=SSDO()detector.fit(X_train,Y_train)# predictlabels=detector.predict(X_test)Similarly, the probability of each point inX_testbeing normal or anomalous can also be computed:probabilities=detector.predict_proba(X_test,method='squash')Sometimes we are interested in detecting anomalies in the training data (e.g., when we are doing a post-mortem analysis):# traindetector=SSDO()detector.fit(X_train,Y_train)# predictlabels=detector.labels_Unsupervised anomaly detection:Unsupervised anomaly detectors do not make use of label information (user feedback) when detecting anomalies in a dataset. Given a dataset with attributesXand labelsY, the unsupervised detectors are trained using onlyX.
Theanomatoolspackage implements two recent semi-supervised anomaly detection algorithms:ThekNNO(k-nearest neighbor outlier detection) algorithm computes for each data point the anomaly score as the distance to its k-nearest neighbor in the dataset [3].TheiNNE(isolation nearest neighbor ensembles) algorithm computes for each data point the anomaly score roughly based on how isolation the point is from the rest of the data [4].Given a training datasetX_trainwith labelsY_train, and a test datasetX_test, the algorithms are applied as follows:fromanomatools.modelsimportkNNO,iNNE# traindetector=kNNO()detector.fit(X_train,Y_train)# predictlabels=detector.predict(X_test)Package structureThe anomaly detection algorithms are located in:anomatools/models/For further examples of how to use the algorithms see the notebooks:anomatools/notebooks/DependenciesTheanomatoolspackage requires the following python packages to be installed:Python 3NumpyScipyScikit-learnContactContact the author of the package:vincent.vercruyssen@kuleuven.beReferences[1] Vercruyssen, V., Meert, W., Verbruggen, G., Maes, K., Bäumer, R., Davis, J. (2018)Semi-Supervised Anomaly Detection with an Application to Water Analytics.IEEE International Conference on Data Mining (ICDM), Singapore. p527--536.[2] Vercruyssen, V., Meert, W., Davis, J. (2020)Transfer Learning for Anomaly Detection through Localized and Unsupervised Instance Selection.AAAI Conference on Artificial Intelligence, New York.