Package: pytorch-ignite
・・TL;DRIgnite is a high-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.Click on the image to see complete codeFeaturesLess code than pure PyTorchwhile ensuring maximum control and simplicityLibrary approach and no program's control inversion -Use ignite where and when you needExtensible API for metrics, experiment managers, and other componentsTable of ContentsTable of ContentsWhy Ignite?Simplified training and validation loopPower of Events & HandlersExecute any number of functions whenever you wishBuilt-in events filteringStack events to share some actionsCustom events to go beyond standard eventsOut-of-the-box metricsInstallationNightly releasesDocker ImagesUsing pre-built imagesGetting StartedDocumentationAdditional MaterialsExamplesTutorialsReproducible Training ExamplesCommunicationUser feedbackContributingProjects using IgniteCiting IgniteAbout the team & DisclaimerWhy Ignite?Ignite is alibrarythat provides three high-level features:Extremely simple engine and event systemOut-of-the-box metrics to easily evaluate modelsBuilt-in handlers to compose training pipeline, save artifacts and log parameters and metricsSimplified training and validation loopNo more codingfor/whileloops on epochs and iterations. Users instantiate engines and run them.Examplefromignite.engineimportEngine,Events,create_supervised_evaluatorfromignite.metricsimportAccuracy# Setup training engine:deftrain_step(engine,batch):# Users can do whatever they need on a single iteration# Eg. forward/backward pass for any number of models, optimizers, etc# ...trainer=Engine(train_step)# Setup single model evaluation engineevaluator=create_supervised_evaluator(model,metrics={"accuracy":Accuracy()})defvalidation():state=evaluator.run(validation_data_loader)# print computed metricsprint(trainer.state.epoch,state.metrics)# Run model's validation at the end of each epochtrainer.add_event_handler(Events.EPOCH_COMPLETED,validation)# Start the trainingtrainer.run(training_data_loader,max_epochs=100)Power of Events & HandlersThe cool thing with handlers is that they offer unparalleled flexibility (compared to, for example, callbacks). Handlers can be any function: e.g. lambda, simple function, class method, etc. Thus, we do not require to inherit from an interface and override its abstract methods which could unnecessarily bulk up your code and its complexity.Execute any number of functions whenever you wishExamplestrainer.add_event_handler(Events.STARTED,lambda_:print("Start training"))# attach handler with args, kwargsmydata=[1,2,3,4]logger=...defon_training_ended(data):print(f"Training is ended. mydata={data}")# User can use variables from another scopelogger.info("Training is ended")trainer.add_event_handler(Events.COMPLETED,on_training_ended,mydata)# call any number of functions on a single eventtrainer.add_event_handler(Events.COMPLETED,lambdaengine:print(engine.state.times))@trainer.on(Events.ITERATION_COMPLETED)deflog_something(engine):print(engine.state.output)Built-in events filteringExamples# run the validation every 5 epochs@trainer.on(Events.EPOCH_COMPLETED(every=5))defrun_validation():# run validation# change some training variable once on 20th epoch@trainer.on(Events.EPOCH_STARTED(once=20))defchange_training_variable():# ...# Trigger handler with customly defined frequency@trainer.on(Events.ITERATION_COMPLETED(event_filter=first_x_iters))deflog_gradients():# ...Stack events to share some actionsExamplesEvents can be stacked together to enable multiple calls:@trainer.on(Events.COMPLETED|Events.EPOCH_COMPLETED(every=10))defrun_validation():# ...Custom events to go beyond standard eventsExamplesCustom events related to backward and optimizer step calls:fromignite.engineimportEventEnumclassBackpropEvents(EventEnum):BACKWARD_STARTED='backward_started'BACKWARD_COMPLETED='backward_completed'OPTIM_STEP_COMPLETED='optim_step_completed'defupdate(engine,batch):# ...loss=criterion(y_pred,y)engine.fire_event(BackpropEvents.BACKWARD_STARTED)loss.backward()engine.fire_event(BackpropEvents.BACKWARD_COMPLETED)optimizer.step()engine.fire_event(BackpropEvents.OPTIM_STEP_COMPLETED)# ...trainer=Engine(update)trainer.register_events(*BackpropEvents)@trainer.on(BackpropEvents.BACKWARD_STARTED)deffunction_before_backprop(engine):# ...Complete snippet is foundhere.Another use-case of custom events:trainer for Truncated Backprop Through Time.Out-of-the-box metricsMetricsfor various tasks:
Precision, Recall, Accuracy, Confusion Matrix, IoU etc, ~20regression metrics.Users can alsocompose their metricswith ease from
existing ones using arithmetic operations or torch methods.Exampleprecision=Precision(average=False)recall=Recall(average=False)F1_per_class=(precision*recall*2/(precision+recall))F1_mean=F1_per_class.mean()# torch mean methodF1_mean.attach(engine,"F1")InstallationFrompip:pipinstallpytorch-igniteFromconda:condainstallignite-cpytorchFrom source:pipinstallgit+https://github.com/pytorch/igniteNightly releasesFrom pip:pipinstall--prepytorch-igniteFrom conda (this suggests to installpytorch nightly releaseinstead of stable
version as dependency):condainstallignite-cpytorch-nightlyDocker ImagesUsing pre-built imagesPull a pre-built docker image fromour Docker Huband run it with docker v19.03+.dockerrun--gpusall-it-v$PWD:/workspace/project--network=host--shm-size16Gpytorchignite/base:latest/bin/bashList of available pre-built imagesBasepytorchignite/base:latestpytorchignite/apex:latestpytorchignite/hvd-base:latestpytorchignite/hvd-apex:latestpytorchignite/msdp-apex:latestVision:pytorchignite/vision:latestpytorchignite/hvd-vision:latestpytorchignite/apex-vision:latestpytorchignite/hvd-apex-vision:latestpytorchignite/msdp-apex-vision:latestNLP:pytorchignite/nlp:latestpytorchignite/hvd-nlp:latestpytorchignite/apex-nlp:latestpytorchignite/hvd-apex-nlp:latestpytorchignite/msdp-apex-nlp:latestFor more details, seehere.Getting StartedFew pointers to get you started:Quick Start Guide: Essentials of getting a project up and runningConcepts of the library: Engine, Events & Handlers, State, MetricsFull-featured template examples (coming soon)DocumentationStable API documentation and an overview of the library:https://pytorch.org/ignite/Development version API documentation:https://pytorch.org/ignite/master/FAQ,"Questions on Github"and"Questions on Discuss.PyTorch".Project's RoadmapAdditional MaterialsDistributed Training Made Easy with PyTorch-IgnitePyTorch Ecosystem Day 2021 Breakout session presentationTutorial blog post about PyTorch-Ignite8 Creators and Core Contributors Talk About Their Model Training Libraries From PyTorch EcosystemIgnite Posters from Pytorch Developer Conferences:202120192018ExamplesTutorialsText Classification using Convolutional Neural
NetworksVariational Auto
EncodersConvolutional Neural Networks for Classifying Fashion-MNIST
DatasetTraining Cycle-GAN on Horses to
Zebras with Nvidia/Apex-logs on W&BAnother training Cycle-GAN on Horses to
Zebras with Native Torch CUDA AMP-logs on W&BFinetuning EfficientNet-B0 on
CIFAR100Hyperparameters tuning with
AxBasic example of LR finder on
MNISTBenchmark mixed precision training on Cifar100:
torch.cuda.amp vs nvidia/apexMNIST training on a single
TPUCIFAR10 Training on multiple TPUsBasic example of handlers
time profiling on MNIST training exampleReproducible Training ExamplesInspired bytorchvision/references,
we provide several reproducible baselines for vision tasks:ImageNet- logs on Ignite Trains server coming soon ...Pascal VOC2012- logs on Ignite Trains server coming soon ...Features:Distributed training: native or horovod and usingPyTorch native AMPCode-Generator applicationThe easiest way to create your training scripts with PyTorch-Ignite:https://code-generator.pytorch-ignite.ai/CommunicationGitHub issues: questions, bug reports, feature requests, etc.Discuss.PyTorch, category "Ignite".PyTorch-Ignite Discord Server: to chat with the communityGitHub Discussions: general library-related discussions, ideas, Q&A, etc.User feedbackWe have created a form for"user feedback". We
appreciate any type of feedback, and this is how we would like to see our
community:If you like the project and want to say thanks, this the right
place.If you do not like something, please, share it with us, and we can
see how to improve it.Thank you!ContributingPlease see thecontribution guidelinesfor more information.As always, PRs are welcome :)Projects using IgniteResearch papersBatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active LearningA Model to Search for Synthesizable MoleculesLocalised Generative FlowsExtracting T Cell Function and Differentiation Characteristics from the Biomedical LiteratureVariational Information Distillation for Knowledge TransferXPersona: Evaluating Multilingual Personalized ChatbotCNN-CASS: CNN for Classification of Coronary Artery Stenosis Score in MPR ImagesBridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware DialogAdversarial Decomposition of Text RepresentationUncertainty Estimation Using a Single Deep Deterministic Neural NetworkDeepSphere: a graph-based spherical CNNNorm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality AssessmentUnified Quality Assessment of In-the-Wild Videos with Mixed Datasets TrainingDeep Signature TransformsNeural CDEs for Long Time-Series via the Log-ODE MethodVolumetric Grasping NetworkMood Classification using Listening DataDeterministic Uncertainty Estimation (DUE)PyTorch-Hebbian: facilitating local learning in a deep learning frameworkStochastic Weight Matrix-Based Regularization Methods for Deep Neural NetworksLearning explanations that are hard to varyThe role of disentanglement in generalisationA Probabilistic Programming Approach to Protein Structure SuperpositionPadChest: A large chest x-ray image dataset with multi-label annotated reportsBlog articles, tutorials, booksState-of-the-Art Conversational AI with Transfer LearningTutorial on Transfer Learning in NLP held at NAACL 2019Deep-Reinforcement-Learning-Hands-On-Second-Edition, published by PacktOnce Upon a Repository: How to Write Readable, Maintainable Code with PyTorchThe Hero Rises: Build Your Own SSDUsing Optuna to Optimize PyTorch Ignite HyperparametersPyTorch Ignite - Classifying Tiny ImageNet with EfficientNetToolkitsProject MONAI - AI Toolkit for Healthcare ImagingDeepSeismic - Deep Learning for Seismic Imaging and InterpretationNussl - a flexible, object-oriented Python audio source separation libraryPyTorch Adapt - A fully featured and modular domain adaptation librarygnina-torch: PyTorch implementation of GNINA scoring functionOthersImplementation of "Attention is All You Need" paperImplementation of DropBlock: A regularization method for convolutional networks in PyTorchKaggle Kuzushiji Recognition: 2nd place solutionUnsupervised Data Augmentation experiments in PyTorchHyperparameters tuning with OptunaLogging with ChainerUIFixMatch experiments in PyTorch and Ignite (CTA dataaug policy)Kaggle Birdcall Identification Competition: 1st place solutionLogging with Aim - An open-source experiment trackerSee other projects at"Used by"If your project implements a paper, represents other use-cases not
covered in our official tutorials, Kaggle competition's code, or just
your code presents interesting results and uses Ignite. We would like to
add your project to this list, so please send a PR with brief
description of the project.Citing IgniteIf you use PyTorch-Ignite in a scientific publication, we would appreciate citations to our project.@misc{pytorch-ignite,
  author = {V. Fomin and J. Anmol and S. Desroziers and J. Kriss and A. Tejani},
  title = {High-level library to help with training neural networks in PyTorch},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/pytorch/ignite}},
}About the team & DisclaimerPyTorch-Ignite is aNumFOCUS Affiliated Project, operated and maintained by volunteers in the PyTorch community in their capacities as individuals
(and not as representatives of their employers). See the"About us"page for a list of core contributors. For usage questions and issues, please see the various channelshere. For all other questions and inquiries, please send an email
tocontact@pytorch-ignite.ai.