Package: orion-ml
An open source project from Data to AI Lab at MIT.OrionA machine learning library for unsupervised time series anomaly detection.Important Links:computer:WebsiteCheck out the Sintel Website for more information about the project.:book:DocumentationQuickstarts, User and Development Guides, and API Reference.:star:TutorialsCheckout our notebooks:octocat:RepositoryThe link to the Github Repository of this library.:scroll:LicenseThe repository is published under the MIT License.CommunityJoin our Slack Workspace for announcements and discussions.OverviewOrion is a machine learning library built forunsupervised time series anomaly detection. With a given time series data, we provide a number of “verified” ML pipelines (a.k.a Orion pipelines) that identify rare patterns and flag them for expert review.The library makes use of a number ofautomated machine learningtools developed underData to AI Lab at MIT.Read about using an Orion pipeline on NYC taxi dataset in a blog series:Part 1: Learn about unsupervised time series anomaly detectionPart 2: Learn how we use GANs to solving the problem?Part 3: How does one evaluate anomaly detection pipelines?Notebooks:DiscoverOrionthrough colab by launching ournotebooks!QuickstartInstall with pipThe easiest and recommended way to installOrionis usingpip:pipinstallorion-mlThis will pull and install the latest stable release fromPyPi.In the following example we show how to use one of theOrion Pipelines.Fit an Orion pipelineWe will load a demo data for this example:fromorion.dataimportload_signaltrain_data=load_signal('S-1-train')train_data.head()which should show a signal withtimestampandvalue.timestamp     value
0  1222819200 -0.366359
1  1222840800 -0.394108
2  1222862400  0.403625
3  1222884000 -0.362759
4  1222905600 -0.370746In this example we useaerpipeline and set some hyperparameters (in this case training epochs as 5).fromorionimportOrionhyperparameters={'orion.primitives.aer.AER#1':{'epochs':5,'verbose':True}}orion=Orion(pipeline='aer',hyperparameters=hyperparameters)orion.fit(train_data)Detect anomalies using the fitted pipelineOnce it is fitted, we are ready to use it to detect anomalies in our incoming time series:new_data=load_signal('S-1-new')anomalies=orion.detect(new_data):warning: Depending on your system and the exact versions that you might have installed someWARNINGSmay be printed. These can be safely ignored as they do not interfere with the proper behavior of the pipeline.The output of the previous command will be apandas.DataFramecontaining a table of detected anomalies:start         end  severity
0  1402012800  1403870400  0.122539LeaderboardIn every release, we run Orion benchmark. We maintain an up-to-date leaderboard with the current scoring of the verified pipelines according to the benchmarking procedure.We run the benchmark on12datasets with their known grounth truth. We record the score of the pipelines on each datasets. To compute the leaderboard table, we showcase the number of wins each pipeline has over the ARIMA pipeline.PipelineOutperforms ARIMAAER11TadGAN7LSTM Dynamic Thresholding8LSTM Autoencoder7Dense Autoencoder7VAE6LNN7Matrix Profile5GANF5Azure0You can find the scores of each pipeline on every signal recorded in thedetails Google Sheets document. The summarized results can also be browsed in the followingsummary Google Sheets document.ResourcesAdditional resources that might be of interest:Learn aboutbenchmarking pipelines.Read aboutpipeline evaluation.Find out more aboutTadGAN.CitationIf you useAERfor your research, please consider citing the following paper:Lawrence Wong, Dongyu Liu, Laure Berti-Equille, Sarah Alnegheimish, Kalyan Veeramachaneni.AER: Auto-Encoder with Regression for Time Series Anomaly Detection.@inproceedings{wong2022aer,
  title={AER: Auto-Encoder with Regression for Time Series Anomaly Detection},
  author={Wong, Lawrence and Liu, Dongyu and Berti-Equille, Laure and Alnegheimish, Sarah and Veeramachaneni, Kalyan},
  booktitle={2022 IEEE International Conference on Big Data (IEEE BigData)},
  pages={1152-1161},
  doi={10.1109/BigData55660.2022.10020857},
  organization={IEEE},
  year={2022}
}If you useTadGANfor your research, please consider citing the following paper:Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, Kalyan Veeramachaneni.TadGAN - Time Series Anomaly Detection Using Generative Adversarial Networks.@inproceedings{geiger2020tadgan,
  title={TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks},
  author={Geiger, Alexander and Liu, Dongyu and Alnegheimish, Sarah and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
  booktitle={2020 IEEE International Conference on Big Data (IEEE BigData)},
  pages={33-43},
  doi={10.1109/BigData50022.2020.9378139},
  organization={IEEE},
  year={2020}
}If you useOrionwhich is part of theSintelecosystem for your research, please consider citing the following paper:Sarah Alnegheimish, Dongyu Liu, Carles Sala, Laure Berti-Equille, Kalyan Veeramachaneni.Sintel: A Machine Learning Framework to Extract Insights from Signals.@inproceedings{alnegheimish2022sintel,
  title={Sintel: A Machine Learning Framework to Extract Insights from Signals},
  author={Alnegheimish, Sarah and Liu, Dongyu and Sala, Carles and Berti-Equille, Laure and Veeramachaneni, Kalyan},  
  booktitle={Proceedings of the 2022 International Conference on Management of Data},
  pages={1855–1865},
  numpages={11},
  publisher={Association for Computing Machinery},
  doi={10.1145/3514221.3517910},
  series={SIGMOD '22},
  year={2022}
}History0.6.0 - 2024-2-13Support for python 3.10 and 3.11Issues resolvedUpdatetest_corefile –Issue #507by @sarahmishupdate ARIMA primitive and pipeline –Issue #503by @sarahmishUpdate Dependency –Issue #497&Issue #499by @sarahmishUpdate Python for Dependency Test –Issue #484by @sarahmishAdd python 3.10 and 3.11 & drop 3.6 and 3.7 –Issue #477by @sarahmishLNN Pipeline –Issue #475by @sarahmish0.5.2 - 2023-10-19Support for python 3.9 and new Matrix Profile pipelineIssues resolvedSend pipeline names in benchmark arguments –Issue #466by @sarahmishAdd continuation parameter for benchmark –Issue #464by @sarahmishFix references in documentation –Issue #453by @sarahmishUpdate documentation –Issue #448by @sarahmishAdd matrix profiling method –Issue #446by @sarahmishSupport python 3.9 –Issue #408by @sarahmish0.5.1 - 2023-08-16This version introduces a new dataset to the benchmark.Issues resolvedAdd UCR dataset to the benchmark –Issue #443by @sarahmishdocker image build failed –Issue #439by @sarahmishEdit interval settings inazurepipeline –Issue #436by @sarahmish0.5.0 - 2023-05-23This version usesml-starspackage instead ofmlprimitives.Issues resolvedMigrate to ml-stars –Issue #418by @sarahmishUpdatingbest_costinfind_anomaliesprimitive –Issue #403by @sarahmishRetirelstm_dynamic_threshold_gpuandlstm_autoencoder_gpupipeline maintenance –Issue #373by @sarahmishTypo in xlsxwriter dependency specification –Issue #394by @sarahmishorion.evaluateuses fails when fitting –Issue #384by @sarahmishAER pipeline with visualization option –Issue #379by @sarahmish0.4.1 - 2023-01-31Issues resolvedMove VAE from sandbox to verified –Issue #377by @sarahmishPinopencv–Issue #372by @sarahmishPinscikit-learn–Issue #367by @sarahmishFix VAE documentation –Issue #360by @sarahmish0.4.0 - 2022-11-08This version introduces several new enhancements:Support to python 3.8Migrating to Tensorflow 2.0New pipeline, namelyVAE, a Variational AutoEncoder model.Issues resolvedAdd python 3.8 –Issue #342by @sarahmishVAE (Variational Autoencoders) pipeline implementation –Issue #349by @dyuliuAdd masking option forregression_errors–Issue #352by @dyuliuChanges in TadGAN for tensorflow 2.0 –Issue #161by @lcwong0928Add an automatic dependency checker –Issue #320by @sarahmishTadGANbatch_sizecannot be changed –Issue #313by @sarahmish0.3.2 - 2022-07-04This version fixes some of the issues inaer,ae, andtadganpipelines.Issues resolvedFix AER model predict error after loading –Issue #304by @lcwong0928Update AE to work with anywindow_size–Issue #300by @sarahmishUpdated tadgan_viz.json –Issue #292by @Hramir0.3.1 - 2022-04-26This version introduce a new pipeline, namelyAER, an AutoEncoder Regressor model.Issues resolvedAdd AER Model -Issue #286by @lcwong09280.3.0 - 2022-03-31This version deprecates the support ofOrionDBExplorer, which has been migrated tosintel. As a result,Orionno longer requires
mongoDB as a dependency.Issues resolvedUpdate dependency  -Issue #283by @sarahmishGeneral housekeeping  -Issue #278by @sarahmishFix tutorial testing issue -Issue #276by @sarahmishMigrate OrionExplorer to Sintel -Issue #275by @dyuliuLSTM viz JSON pipeline added -Issue #271by @Hramir0.2.1 - 2022-02-18This version introduces improvements and more testing.Issues resolvedAdjusting builds for TadGAN -Issue #261by @sarahmishTesting tutorials, dependencies, and OS -Issue #251by @sarahmish0.2.0 - 2021-10-11This version supports multivariate timeseries as input. In addition to minor improvements
and maintenance.Issues resolvedsetuptoolsno longer supportslib2to3breakingmongoengine-Issue #252by @sarahmishSupporting multivariate input -Issue #248by @sarahmishTadGAN pipeline with visualization option -Issue #240by @sarahmishSupport saving absolute path for add_signals and add_signal when using dbExplorer -Issue #202by @sarahmishdynamic scalability of TadGAN primitive based onwindow_size-Issue #87by @sarahmish0.1.7 - 2021-05-04This version adds new features to the benchmark function where users can now save pipelines, view results as they are being calculated, and allow a single evaluation to be compared multiple times.Issues resolvedDask issues in benchmark function & improvements -Issue #225by @sarahmishNumerical overflow when using contextual metrics -Issue #212by @kronerte0.1.6 - 2021-03-08This version introduces two new pipelines: LSTM AE and Dense AE.
In addition to minor improvements, a bit of code refactoring took place to introduce
a new primtive:reconstruction_errors.Issues resolvedComparison of DTW library performance -Issue #205by @sarahmishNot able to pickle dump tadgan pipeline -Issue #200by @sarahmishNew pipeline LSTM and Dense autoencoders -Issue #194by @sarahmishReadme -Issue #192by @pvk-developerUnable to launch cli -Issue #186by @sarahmishbullet points not formatted correctly in index.rst -Issue #178by @micahjsmithUpdate notebooks -Issue #176by @sarahmishInaccuracy in README.md file in orion/evaluation/ -Issue #157by @sarahmishDockerfile -- docker does not find orion primitives automatically -Issue #155by @sarahmishPrimitive documentation -Issue #151by @sarahmishVariable name inconsistency in tadgan -Issue #150by @sarahmishSync leaderboard tables betweenBENCHMARK.mdand the docs -Issue #148by @sarahmish0.1.5 - 2020-12-25This version includes the new style of documentation and a revamp of theREADME.md. In addition to some minor improvements
in the benchmark code and primitives. This release includes the transfer oftadganpipeline toverified.Issues resolvedLink with google colab -Issue #144by @sarahmishAddtimeseries_anomaliesunittests -Issue #136by @sarahmishUpdatefind_sequencesin converting series to arrays -Issue #135by @sarahmishDefinition of error/critic smooth window in score anomalies primitive -Issue #132by @sarahmishTrain-test split in benchmark enhancement -Issue #130by @sarahmish0.1.4 - 2020-10-16Minor enhancements to benchmarkLoad ground truth before try-catch -Issue #124by @sarahmishConverting timestamp to datetime in Azure primitive -Issue #123by @sarahmishBenchmark exceptions -Issue #120by @sarahmish0.1.3 - 2020-09-29New benchmark and Azure primitive.Implement a benchmarking function new feature -Issue #94by @sarahmishAdd azure anomaly detection as primitive new feature -Issue #97by @sarahmishCritic and reconstruction error combination -Issue #99by @sarahmishFixed threshold forfind_anomalies-Issue #101by @sarahmishAdd an option to have window size and window step size as percentages of error size -Issue #102by @sarahmishOrganize pipelines into verified and sandbox -Issue #105by @sarahmishGround truth parameter name enhancement -Issue #114by @sarahmishAdd benchmark dataset list and parameters to s3 bucket enhancement -Issue #118by @sarahmish0.1.2 - 2020-07-03New Evaluation sub-package and refactor TadGAN.Two bugs when saving signalrun if there is no event detected -Issue #92by @dyuliuFile encoding/decoding issues aboutREADME.mdandHISTORY.md-Issue #88by @dyuliuFix bottle neck ofscore_anomalyin Cyclegan primitive -Issue #86by @dyuliuAdjustepochmeaning in Cyclegan primitive -Issue #85by @sarahmishRename evaluation to benchmark and metrics to evaluation -Issue #83by @sarahmishScoring function for intervals of size one -Issue #76by @sarahmish0.1.1 - 2020-05-11New class and function based interfaces.Implement the Orion Class -Issue #79by @csalaImplement new functional interface -Issue #80by @csala0.1.0 - 2020-04-23First Orion release to PyPI:https://pypi.org/project/orion-ml/