Package: anomalib
A library for benchmarking, developing and deploying deep learning anomaly detection algorithmsKey Features•Getting Started•Docs•LicenseIntroductionAnomalib is a deep learning library that aims to collect state-of-the-art anomaly detection algorithms for benchmarking on both public and private datasets. Anomalib provides several ready-to-use implementations of anomaly detection algorithms described in the recent literature, as well as a set of tools that facilitate the development and implementation of custom models. The library has a strong focus on image-based anomaly detection, where the goal of the algorithm is to identify anomalous images, or anomalous pixel regions within images in a dataset. Anomalib is constantly updated with new algorithms and training/inference extensions, so keep checking!Key featuresThe largest public collection of ready-to-use deep learning anomaly detection algorithms and benchmark datasets.PyTorch Lightningbased model implementations to reduce boilerplate code and limit the implementation efforts to the bare essentials.All models can be exported toOpenVINOIntermediate Representation (IR) for accelerated inference on intel hardware.A set ofinference toolsfor quick and easy deployment of the standard or custom anomaly detection models.Getting StartedFollowing is a guide on how to get started withanomalib. For more details, look at theDocumentation.Jupyter NotebooksFor getting started with a Jupyter Notebook, please refer to theNotebooksfolder of this repository. Additionally, you can refer to a few created by the community:PyPI InstallYou can get started withanomalibby just using pip.pipinstallanomalibLocal InstallIt is highly recommended to use virtual environment when installing anomalib. For instance, withanaconda,anomalibcould be installed as,yes|condacreate-nanomalib_envpython=3.10
condaactivateanomalib_env
gitclonehttps://github.com/openvinotoolkit/anomalib.gitcdanomalib
pipinstall-e.TrainingBy defaultpython tools/train.pyrunsPADIMmodel onleathercategory from theMVTec AD(CC BY-NC-SA 4.0)dataset.pythontools/train.py# Train PADIM on MVTec AD leatherTraining a model on a specific dataset and category requires further configuration. Each model has its own configuration
file,config.yaml, which contains data, model and training configurable parameters. To train a specific model on a specific dataset and
category, the config file is to be provided:pythontools/train.py--config<path/to/model/config.yaml>For example, to trainPADIMyou can usepythontools/train.py--configsrc/anomalib/models/padim/config.yamlAlternatively, a model name could also be provided as an argument, where the scripts automatically finds the corresponding config file.pythontools/train.py--modelpadimwhere the currently available models are:CFACFlowDFKDEDFMDRAEMEfficientAdFastFlowGANomalyPADIMPatchCoreReverse DistillationSTFPMFeature extraction & (pre-trained) backbonesThe pre-trained backbones come fromPyTorch Image Models (timm), which are wrapped byFeatureExtractor.For more information, please check our documentation or thesection about feature extraction in "Getting Started with PyTorch Image Models (timm): A Practitioner’s Guide".Tips:Papers With Code has an interface to easily browse models available in timm:https://paperswithcode.com/lib/timmYou can also find them with the functiontimm.list_models("resnet*", pretrained=True)The backbone can be set in the config file, two examples below.model:name:cflowbackbone:wide_resnet50_2pre_trained:trueCustom DatasetIt is also possible to train on a custom folder dataset. To do so,datasection inconfig.yamlis to be modified as follows:dataset:name:<name-of-the-dataset>format:folderpath:<path/to/folder/dataset>normal_dir:normal# name of the folder containing normal images.abnormal_dir:abnormal# name of the folder containing abnormal images.normal_test_dir:null# name of the folder containing normal test images.task:segmentation# classification or segmentationmask:<path/to/mask/annotations>#optionalextensions:nullsplit_ratio:0.2# ratio of the normal images that will be used to create a test splitimage_size:256train_batch_size:32test_batch_size:32num_workers:8transform_config:train:nullval:nullcreate_validation_set:truetiling:apply:falsetile_size:nullstride:nullremove_border_count:0use_random_tiling:Falserandom_tile_count:16InferenceAnomalib includes multiple tools, including Lightning, Gradio, and OpenVINO inferencers, for performing inference with a trained model.The following command can be used to run PyTorch Lightning inference from the command line:pythontools/inference/lightning_inference.py-hAs a quick example:pythontools/inference/lightning_inference.py\--configsrc/anomalib/models/padim/config.yaml\--weightsresults/padim/mvtec/bottle/run/weights/model.ckpt\--inputdatasets/MVTec/bottle/test/broken_large/000.png\--outputresults/padim/mvtec/bottle/imagesExample OpenVINO Inference:pythontools/inference/openvino_inference.py\--weightsresults/padim/mvtec/bottle/run/openvino/model.bin\--metadataresults/padim/mvtec/bottle/run/openvino/metadata.json\--inputdatasets/MVTec/bottle/test/broken_large/000.png\--outputresults/padim/mvtec/bottle/imagesEnsure that you provide path tometadata.jsonif you want the normalization to be applied correctly.You can also use Gradio Inference to interact with the trained models using a UI. Refer to ourguidefor more details.A quick example:pythontools/inference/gradio_inference.py\--weightsresults/padim/mvtec/bottle/run/weights/model.ckptExporting Model to ONNX or OpenVINO IRIt is possible to export your model to ONNX or OpenVINO IRIf you want to export your PyTorch model to an OpenVINO model, ensure thatexport_modeis set to"openvino"in the respective modelconfig.yaml.optimization:export_mode:"openvino"# options: openvino, onnxHyperparameter OptimizationTo run hyperparameter optimization, use the following command:pythontools/hpo/sweep.py\--modelpadim--model_config./path_to_config.yaml\--sweep_configtools/hpo/sweep.yamlFor more details refer theHPO DocumentationBenchmarkingTo gather benchmarking data such as throughput across categories, use the following command:pythontools/benchmarking/benchmark.py\--config<relative/absolutepath>/<paramfile>.yamlRefer to theBenchmarking Documentationfor more details.Experiment ManagementAnomablib is integrated with various libraries for experiment tracking such as Comet, tensorboard, and wandb throughpytorch lighting loggers.Below is an example of how to enable logging for hyper-parameters, metrics, model graphs, and predictions on images in the test data-setvisualization:log_images:True# log images to the available loggers (if any)mode:full# options: ["full", "simple"]logging:logger:[comet,tensorboard,wandb]log_graph:TrueFor more information, refer to theLogging DocumentationNote: Set your API Key forComet.mlviacomet_ml.init()in interactive python or simply runexport COMET_API_KEY=<Your API Key>Community Projects1. Web-based Pipeline for Training and InferenceThis project showcases an end-to-end training and inference pipeline build on top of Anomalib. It provides a web-based UI for uploading MVTec style datasets and training them on the available Anomalib models. It also has sections for calling inference on individual images as well as listing all the images with their predictions in the database.You can view the project onGithubFor more details see theDiscussion forumDatasetsanomalibsupports MVTec AD(CC BY-NC-SA 4.0)and BeanTech(CC-BY-SA)for benchmarking andfolderfor custom dataset training/inference.MVTec AD DatasetMVTec AD dataset is one of the main benchmarks for anomaly detection, and is released under the
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License(CC BY-NC-SA 4.0).Note: These metrics are collected with image size of 256 and seed42. This common setting is used to make model comparisons fair.Image-Level AUCModelAvgCarpetGridLeatherTileWoodBottleCableCapsuleHazelnutMetal NutPillScrewToothbrushTransistorZipperEfficientAdPDN-S0.9820.9821.0000.9971.0000.9861.0000.9520.9500.9520.9790.9870.9600.9970.9990.994EfficientAdPDN-M0.9750.9720.9981.0000.9990.9840.9910.9450.9570.9480.9890.9260.9751.0000.9650.971PatchCoreWide ResNet-500.9800.9840.9591.0001.0000.9891.0000.9900.9821.0000.9940.9240.9600.9331.0000.982PatchCoreResNet-180.9730.9700.9471.0000.9970.9971.0000.9860.9651.0000.9910.9160.9430.9310.9960.953CFlowWide ResNet-500.9620.9860.9621.0000.9990.9931.00.8930.9451.00.9950.9240.9080.8970.9430.984CFAWide ResNet-500.9560.9780.9610.9900.9990.9940.9980.9790.8721.0000.9950.9460.7031.0000.9570.967CFAResNet-180.9300.9530.9470.9991.0001.0000.9910.9470.8580.9950.9320.8870.6250.9940.8950.919PaDiMWide ResNet-500.9500.9950.9421.0000.9740.9930.9990.8780.9270.9640.9890.9390.8450.9420.9760.882PaDiMResNet-180.8910.9450.8570.9820.9500.9760.9940.8440.9010.7500.9610.8630.7590.8890.9200.780DFMWide ResNet-500.9430.8550.7840.9970.9950.9750.9990.9690.9240.9780.9390.9620.8730.9690.9710.961DFMResNet-180.9360.8170.7360.9930.9660.9771.0000.9560.9440.9940.9220.9610.890.9690.9390.969STFPMWide ResNet-500.8760.9570.9770.9810.9760.9390.9870.8780.7320.9950.9730.6520.8250.5000.8750.899STFPMResNet-180.8930.9540.9820.9890.9490.9610.9790.8380.7590.9990.9560.7050.8350.9970.8530.645DFKDEWide ResNet-500.7740.7080.4220.9050.9590.9030.9360.7460.8530.7360.6870.7490.5740.6970.8430.892DFKDEResNet-180.7620.6460.5770.6690.9650.8630.9510.7510.6980.8060.7290.6070.6940.7670.8390.866GANomaly0.4210.2030.4040.4130.4080.7440.2510.4570.6820.5370.2700.4720.2310.3720.4400.434Pixel-Level AUCModelAvgCarpetGridLeatherTileWoodBottleCableCapsuleHazelnutMetal NutPillScrewToothbrushTransistorZipperCFAWide ResNet-500.9830.9800.9540.9890.9850.9740.9890.9880.9890.9850.9920.9880.9790.9910.9770.990CFAResNet-180.9790.9700.9730.9920.9780.9640.9860.9840.9870.9870.9810.9810.9730.9900.9640.978PatchCoreWide ResNet-500.9800.9880.9680.9910.9610.9340.9840.9880.9880.9870.9890.9800.9890.9880.9810.983PatchCoreResNet-180.9760.9860.9550.9900.9430.9330.9810.9840.9860.9860.9860.9740.9910.9880.9740.983CFlowWide ResNet-500.9710.9860.9680.9930.9680.9240.9810.9550.9880.9900.9820.9830.9790.9850.8970.980PaDiMWide ResNet-500.9790.9910.9700.9930.9550.9570.9850.9700.9880.9850.9820.9660.9880.9910.9760.986PaDiMResNet-180.9680.9840.9180.9940.9340.9470.9830.9650.9840.9780.9700.9570.9780.9880.9680.979EfficientAdPDN-S0.9600.9630.9370.9760.9070.8680.9830.9830.9800.9760.9780.9860.9850.9620.9560.961EfficientAdPDN-M0.9570.9480.9370.9760.9060.8670.9760.9860.9570.9770.9840.9780.9860.9640.9470.960STFPMWide ResNet-500.9030.9870.9890.9800.9660.9560.9660.9130.9560.9740.9610.9460.9880.1780.8070.980STFPMResNet-180.9510.9860.9880.9910.9460.9490.9710.8980.9620.9810.9420.8780.9830.9830.8380.972Image F1 ScoreModelAvgCarpetGridLeatherTileWoodBottleCableCapsuleHazelnutMetal NutPillScrewToothbrushTransistorZipperPatchCoreWide ResNet-500.9760.9710.9741.0001.0000.9671.0000.9680.9821.0000.9840.9400.9430.9381.0000.979PatchCoreResNet-180.9700.9490.9461.0000.980.9921.0000.9780.9691.0000.9890.9400.9320.9350.9740.967EfficientAdPDN-S0.9700.9661.0000.9951.0000.9751.0000.9070.9560.8970.9780.9820.9440.9840.9880.983EfficientAdPDN-M0.9660.9770.9911.0000.9940.9670.9840.9220.9690.8840.9840.9520.9551.0000.9290.979CFAWide ResNet-500.9620.9610.9570.9950.9940.9830.9840.9620.9461.0000.9840.9520.8551.0000.9070.975CFAResNet-180.9460.9560.9460.9731.0001.0000.9830.9070.9380.9960.9580.9200.8580.9840.7950.949CFlowWide ResNet-500.9440.9720.9321.0000.9880.9671.0000.8320.9391.0000.9790.9240.9710.8700.8180.967PaDiMWide ResNet-500.9510.9890.9301.0000.9600.9830.9920.8560.9820.9370.9780.9460.8950.9520.9140.947PaDiMResNet-180.9160.9300.8930.9840.9340.9520.9760.8580.9600.8360.9740.9320.8790.9230.7960.915DFMWide ResNet-500.9500.9150.8700.9950.9880.9600.9920.9390.9650.9710.9420.9560.9060.9660.9140.971DFMResNet-180.9430.8950.8710.9780.9580.9001.0000.9350.9650.9660.9420.9560.9140.9660.8680.964STFPMWide ResNet-500.9260.9730.9730.9740.9650.9290.9760.8530.9200.9720.9740.9220.8840.8330.8150.931STFPMResNet-180.9320.9610.9820.9890.9300.9510.9840.8190.9180.9930.9730.9180.8870.9840.7900.908DFKDEWide ResNet-500.8750.9070.8440.9050.9450.9140.9460.7900.9140.8170.8940.9220.8550.8450.7220.910DFKDEResNet-180.8720.8640.8440.8540.9600.8980.9420.7930.9080.8270.8940.9160.8590.8530.7560.916GANomaly0.8340.8640.8440.8520.8360.8630.8630.7600.9050.7770.8940.9160.8530.8330.5710.881ReferenceIf you use this library and love it, use this to cite it 🤗@misc{anomalib,
      title={Anomalib: A Deep Learning Library for Anomaly Detection},
      author={Samet Akcay and
              Dick Ameln and
              Ashwin Vaidya and
              Barath Lakshmanan and
              Nilesh Ahuja and
              Utku Genc},
      year={2022},
      eprint={2202.08341},
      archivePrefix={arXiv},
      primaryClass={cs.CV}}ContributingFor those who would like to contribute to the library, seeCONTRIBUTING.mdfor details.Thank you to all of the people who have already made a contribution - we appreciate your support!