Package: deepod
DeepODis an open-source python library for Deep Learning-basedOutlier DetectionandAnomaly Detection.DeepODsupports tabular anomaly detection and time-series anomaly detection.DeepOD includes25deep outlier detection / anomaly detection algorithms (in unsupervised/weakly-supervised paradigm). More baseline algorithms will be included later.DeepOD is featured for:Unified APIsacross various algorithms.SOTA modelsincludes reconstruction-, representation-learning-, and self-superivsed-based latest deep learning methods.Comprehensive Testbedthat can be used to directly test different models on benchmark datasets (highly recommend for academic research).Versatilein different data types including tabular and time-series data (DeepOD will support other data types like images, graph, log, trace, etc. in the future, welcome PR :telescope:).Diverse Network Structurescan be plugged into detection models, we now support LSTM, GRU, TCN, Conv, and Transformer for time-series data.  (welcome PR as well :sparkles:)If you are interested in our project, we are pleased to have your stars and forks :thumbsup: :beers: .InstallationThe DeepOD framework can be installed via:pipinstalldeepodinstall a developing version (strongly recommend)gitclonehttps://github.com/xuhongzuo/DeepOD.gitcdDeepODpipinstall.UsagesDirectly use detection models in DeepOD:DeepOD can be used in a few lines of code. This API style is the same withSkleanandPyOD.for tabular anomaly detection:# unsupervised methodsfromdeepod.models.tabularimportDeepSVDDclf=DeepSVDD()clf.fit(X_train,y=None)scores=clf.decision_function(X_test)# weakly-supervised methodsfromdeepod.models.tabularimportDevNetclf=DevNet()clf.fit(X_train,y=semi_y)# semi_y uses 1 for known anomalies, and 0 for unlabeled datascores=clf.decision_function(X_test)# evaluation of tabular anomaly detectionfromdeepod.metricsimporttabular_metricsauc,ap,f1=tabular_metrics(y_test,scores)for time series anomaly detection:# time series anomaly detection methodsfromdeepod.models.time_seriesimportTimesNetclf=TimesNet()clf.fit(X_train)scores=clf.decision_function(X_test)# evaluation of time series anomaly detectionfromdeepod.metricsimportts_metricsfromdeepod.metricsimportpoint_adjustment# execute point adjustment for time series adeval_metrics=ts_metrics(labels,scores)adj_eval_metrics=ts_metrics(labels,point_adjustment(labels,scores))Testbed usage:Testbed contains the whole process of testing an anomaly detection model, including data loading, preprocessing, anomaly detection, and evaluation.Please refer totestbed/testbed/testbed_unsupervised_ad.pyis for testing unsupervised tabular anomaly detection models.testbed/testbed_unsupervised_tsad.pyis for testing unsupervised time-series anomaly detection models.Key arguments:--input_dir: name of the folder that contains datasets (.csv, .npy)--dataset: “FULL” represents testing all the files within the folder, or a list of dataset names using commas to split them (e.g., “10_cover*,20_letter*”)--model: anomaly detection model name--runs: how many times running the detection model, finally report an average performance with standard deviation valuesExample:DownloadADBenchdatasets.modify thedataset_rootvariable as the directory of the dataset.input_diris the sub-folder name of thedataset_root, e.g.,ClassicalorNLP_by_BERT.use the following command in the bashcdDeepODpipinstall.cdtestbedpythontestbed_unsupervised_ad.py--modelDIF--runs5--input_dirADBenchImplemented ModelsTabular Anomaly Detection models:ModelVenueYearTypeTitleDeep SVDDICML2018unsupervisedDeep One-Class Classification[1]REPENKDD2018unsupervisedLearning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection[2]RDPIJCAI2020unsupervisedUnsupervised Representation Learning by Predicting Random Distances[3]RCAIJCAI2021unsupervisedRCA: A Deep Collaborative Autoencoder Approach for Anomaly Detection[4]GOADICLR2020unsupervisedClassification-Based Anomaly Detection for General Data[5]NeuTraLICML2021unsupervisedNeural Transformation Learning for Deep Anomaly Detection Beyond Images[6]ICLICLR2022unsupervisedAnomaly Detection for Tabular Data with Internal Contrastive LearningDIFTKDE2023unsupervisedDeep Isolation Forest for Anomaly DetectionSLADICML2023unsupervisedFascinating Supervisory Signals and Where to Find Them: Deep Anomaly Detection with Scale LearningDevNetKDD2019weakly-supervisedDeep Anomaly Detection with Deviation NetworksPReNetKDD2023weakly-supervisedDeep Weakly-supervised Anomaly DetectionDeep SADICLR2020weakly-supervisedDeep Semi-Supervised Anomaly DetectionFeaWADTNNLS2021weakly-supervisedFeature Encoding with AutoEncoders for Weakly-supervised Anomaly DetectionRoSASIP&M2023weakly-supervisedRoSAS: Deep semi-supervised anomaly detection with contamination-resilient continuous supervisionTime-series Anomaly Detection models:ModelVenueYearTypeTitleTimesNetICLR2023unsupervisedTIMESNET: Temporal 2D-Variation Modeling for General Time Series AnalysisAnomalyTransformerICLR2022unsupervisedAnomaly Transformer: Time Series Anomaly Detection with Association DiscrepancyTranADVLDB2022unsupervisedTranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series DataCOUTAarXiv2022unsupervisedCalibrated One-class Classification for Unsupervised Time Series Anomaly DetectionUSADKDD2020unsupervisedUSAD: UnSupervised Anomaly Detection on Multivariate Time SeriesDIFTKDE2023unsupervisedDeep Isolation Forest for Anomaly DetectionTcnEDTNNLS2021unsupervisedAn Evaluation of Anomaly Detection and Diagnosis in Multivariate Time SeriesDeep SVDD (TS)ICML2018unsupervisedDeep One-Class ClassificationDevNet (TS)KDD2019weakly-supervisedDeep Anomaly Detection with Deviation NetworksPReNet (TS)KDD2023weakly-supervisedDeep Weakly-supervised Anomaly DetectionDeep SAD (TS)ICLR2020weakly-supervisedDeep Semi-Supervised Anomaly DetectionNOTE:For Deep SVDD, DevNet, PReNet, and DeepSAD, we employ network structures that can handle time-series data. These models’ classes have a parameter namednetworkin these models, by changing it, you can use different networks.We currently support ‘TCN’, ‘GRU’, ‘LSTM’, ‘Transformer’, ‘ConvSeq’, and ‘DilatedConv’.CitationIf you use this library in your work, please cite this paper:Hongzuo Xu, Guansong Pang, Yijie Wang and Yongjun Wang, “Deep Isolation Forest for Anomaly Detection,” in IEEE Transactions on Knowledge and Data Engineering, doi: 10.1109/TKDE.2023.3270293.You can also use the BibTex entry below for citation.@ARTICLE{xu2023deep,author={Xu, Hongzuo and Pang, Guansong and Wang, Yijie and Wang, Yongjun},journal={IEEE Transactions on Knowledge and Data Engineering},title={Deep Isolation Forest for Anomaly Detection},year={2023},volume={},number={},pages={1-14},doi={10.1109/TKDE.2023.3270293}}Reference[1]Ruff, Lukas, et al. “Deep one-class classification.” ICML. 2018.[2]Pang, Guansong, et al. “Learning representations of ultrahigh-dimensional data for random distance-based outlier detection”. KDD (pp. 2041-2050). 2018.[3]Wang, Hu, et al. “Unsupervised Representation Learning by Predicting Random Distances”. IJCAI (pp. 2950-2956). 2020.[4]Liu, Boyang, et al. “RCA: A Deep Collaborative Autoencoder Approach for Anomaly Detection”. IJCAI (pp. 1505-1511). 2021.[5]Bergman, Liron, and Yedid Hoshen. “Classification-Based Anomaly Detection for General Data”. ICLR. 2020.[6]Qiu, Chen, et al. “Neural Transformation Learning for Deep Anomaly Detection Beyond Images”. ICML. 2021.