Package: pyodds
PyODDSOfficial Website:pyodds.github.ioPyODDSis an end-to endPythonsystem foroutlierdetectionwithdatabasesupport. PyODDS provides outlier detection algorithms which meet the demands for users in different fields, w/wo data science or machine learning background. PyODDS gives the ability to execute machine learning algorithms in-database without moving data out of the database server or over the network. It also provides access to a wide range of outlier detection algorithms, including statistical analysis and more recent deep learning based approaches.PyODDS is featured for:Full Stack Servicewhich supports operations and maintenances from light-weight SQL based database to back-end machine learning algorithms and makes the throughput speed faster;State-of-the-art Anomaly Detection ApproachesincludingStatistical/Machine Learning/Deep Learningmodels with unified APIs and detailed documentation;Powerful Data Analysis Mechanismwhich supports bothstatic and time-series dataanalysis with flexible time-slice(sliding-window) segmentation.The Full API Reference can be found inhandbook.API Demo:fromutils.import_algorithmimportalgorithm_selection
fromutils.utilitiesimportoutput_performance,connect_server,query_data# connect to the databaseconn,cursor=connect_server(host,user,password)# query data from specific time rangedata=query_data(database_name,table_name,start_time,end_time)# train the anomaly detection algorithmclf=algorithm_selection(algorithm_name)clf.fit(X_train)# get outlier result and scoresprediction_result=clf.predict(X_test)outlierness_score=clf.decision_function(test)#visualize the prediction_resultvisualize_distribution(X_test,prediction_result,outlierness_score)Cite this workYuening Li, Daochen Zha, Na Zou, Xia Hu. "PyODDS: An End-to-End Outlier Detection System"  (Download)Biblatex entry:@article{li2019pyodds,
  author = {Li, Yuening and Zha, Daochen and Zou, Na and Hu, Xia},
  title = {PyODDS: An End-to-End Outlier Detection System},
  year = {2019},
  eprint = {arXiv:1910.02575},
}Quick Startpythondemo.py--ground_truth--visualize_distributionResults are shown asconnecttoTDenginesuccess
Loaddatasetandtable
Loadingcost:0.151061seconds
Loaddatasuccessful
Startprocessing:100%|████████████████████████████████████|10/10[00:00<00:00,14.02it/s]==============================ResultsinAlgorithmdagmmare:
accuracy_score:0.98
precision_score:0.99
recall_score:0.99
f1_score:0.99
roc_auc_score:0.99
processingtime:15.330137seconds==============================connectionisclosedInstallationTo install the package, please use thepipinstallation as follows:pipinstallpyodds
pipinstallgit+git@github.com:datamllab/PyODDS.gitNote:PyODDS is only compatible withPython 3.6and above.Required Dependencies-pandas>=0.25.0
-taos==1.4.15
-tensorflow==2.0.0b1
-numpy>=1.16.4
-seaborn>=0.9.0
-torch>=1.1.0
-luminol==0.4
-tqdm>=4.35.0
-matplotlib>=3.1.1
-scikit_learn>=0.21.3To compile and package the JDBC driver source code, you should have a Java jdk-8 or higher and Apache Maven 2.7 or higher installed. To install openjdk-8 on Ubuntu:sudoapt-getinstallopenjdk-8-jdkTo install Apache Maven on Ubuntu:sudoapt-getinstallmavenTo install the TDengine as the back-end database service, please refer tothis instruction.To enable the Python client APIs for TDengine, please followthis handbook.To insure the locale in config file is valid:sudolocale-gen"en_US.UTF-8"exportLC_ALL="en_US.UTF-8"localeTo start the service after installation, in a terminal, use:taosdImplemented AlgorithmsStatistical Based MethodsMethodsAlgorithmClass APICBLOFClustering-Based Local Outlier Factor:class:algo.cblof.CBLOFHBOSHistogram-based Outlier Score:class:algo.hbos.HBOSIFORESTIsolation Forest:class:algo.iforest.IFORESTKNNk-Nearest Neighbors:class:algo.knn.KNNLOFLocal Outlier Factor:class:algo.cblof.CBLOFOCSVMOne-Class Support Vector Machines:class:algo.ocsvm.OCSVMPCAPrincipal Component Analysis:class:algo.pca.PCARobustCovarianceRobust Covariance:class:algo.robustcovariance.RCOVSODSubspace Outlier Detection:class:algo.sod.SODDeep Learning Based MethodsMethodsAlgorithmClass APIautoencoderOutlier detection using replicator neural networks:class:algo.autoencoder.AUTOENCODERdagmmDeep autoencoding gaussian mixture model for unsupervised anomaly detection:class:algo.dagmm.DAGMMTime Serie MethodsMethodsAlgorithmClass APIlstmadLong short term memory networks for anomaly detection in time series:class:algo.lstm_ad.LSTMADlstmencdecLSTM-based encoder-decoder for multi-sensor anomaly detection:class:algo.lstm_enc_dec_axl.LSTMEDluminolLinkedin's luminol:class:algo.luminol.LUMINOLAPIs CheatsheetThe Full API Reference can be found inhandbook.connect_server(hostname,username,password): Connect to Apache backend TDengine Service.query_data(connection,cursor,database_name,table_name,start_time,end_time): Query data from tabletable_namein databasedatabase_namewithin a given time range.algorithm_selection(algorithm_name,contamination): Select an algorithm as detector.fit(X): FitXto detector.predict(X): Predict if instance inXis outlier or not.decision_function(X): Output the anomaly score of instances inX.output_performance(algorithm_name,ground_truth,prediction_result,outlierness_score): Output the prediction result as evaluation matrix inAccuracy,Precision,Recall,F1 Score,ROC-AUC Score,Cost time.visualize_distribution(X,prediction_result,outlierness_score): Visualize the detection result with the the data distribution.visualize_outlierscore(outlierness_score,prediction_result,contamination)Visualize the detection result with the outlier score.LicenseYou may use this software under the MIT License.