Package: lightning-flash
Your PyTorch AI FactoryInstallation•Flash in 3 Steps•Docs•Contribute•Community•Website•LicenseFlash makes complex AI recipes for over 15 tasks across 7 data domains accessible to all.In a nutshell, Flash is the production grade research framework you always dreamed of but didn't have time to build.Getting StartedFrom PyPI:pipinstalllightning-flashSeeour installation guidefor more options.Flash in 3 StepsStep 1. Load your dataAll data loading in Flash is performed via afrom_*classmethod on aDataModule.
To decide whichDataModuleto use and whichfrom_*methods are available, it depends on the task you want to perform.
For example, for image segmentation where your data is stored in folders, you would use thefrom_foldersmethod of theSemanticSegmentationDataclass:fromflash.imageimportSemanticSegmentationDatadm=SemanticSegmentationData.from_folders(train_folder="data/CameraRGB",train_target_folder="data/CameraSeg",val_split=0.1,image_size=(256,256),num_classes=21,)Step 2: Configure your modelOur tasks come loaded with pre-trained backbones and (where applicable) heads.
You can view the available backbones to use with your task usingavailable_backbones.
Once you've chosen one, create the model:fromflash.imageimportSemanticSegmentationprint(SemanticSegmentation.available_heads())# ['deeplabv3', 'deeplabv3plus', 'fpn', ..., 'unetplusplus']print(SemanticSegmentation.available_backbones('fpn'))# ['densenet121', ..., 'xception'] # + 113 modelsprint(SemanticSegmentation.available_pretrained_weights('efficientnet-b0'))# ['imagenet', 'advprop']model=SemanticSegmentation(head="fpn",backbone='efficientnet-b0',pretrained="advprop",num_classes=dm.num_classes)Step 3: Finetune!fromflashimportTrainertrainer=Trainer(max_epochs=3)trainer.finetune(model,datamodule=datamodule,strategy="freeze")trainer.save_checkpoint("semantic_segmentation_model.pt")PyTorch RecipesMake predictions with Flash!Serve in just 2 lines:fromflash.imageimportSemanticSegmentationmodel=SemanticSegmentation.load_from_checkpoint("semantic_segmentation_model.pt")model.serve()or make predictions from raw data directly.fromflashimportTrainertrainer=Trainer(strategy='ddp',accelerator="gpu",gpus=2)dm=SemanticSegmentationData.from_folders(predict_folder="data/CameraRGB")predictions=trainer.predict(model,dm)Flash Training StrategiesTraining strategies are PyTorch SOTA Training Recipes which can be utilized with a given task.Check out thisexamplewhere theImageClassifiersupports 4Meta Learning AlgorithmsfromLearn2Learn.
This is particularly useful if you use this model in production and want to make sure the model adapts quickly to its new environment with minimal labelled data.fromflash.imageimportImageClassifiermodel=ImageClassifier(backbone="resnet18",optimizer=torch.optim.Adam,optimizer_kwargs={"lr":0.001},training_strategy="prototypicalnetworks",training_strategy_kwargs={"epoch_length":10*16,"meta_batch_size":4,"num_tasks":200,"test_num_tasks":2000,"ways":datamodule.num_classes,"shots":1,"test_ways":5,"test_shots":1,"test_queries":15,},)In detail, the following methods are currently implemented:prototypicalnetworks: from Snellet al.2017,Prototypical Networks for Few-shot Learningmaml: from Finnet al.2017,Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networksmetaoptnet: from Leeet al.2019,Meta-Learning with Differentiable Convex Optimizationanil: from Raghuet al.2020,Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLFlash Optimizers / SchedulersWith Flash, swapping among 40+ optimizers and 15+ schedulers recipes are simple. Find the list of available optimizers, schedulers as follows:fromflash.imageimportImageClassifierImageClassifier.available_optimizers()# ['A2GradExp', ..., 'Yogi']ImageClassifier.available_schedulers()# ['CosineAnnealingLR', 'CosineAnnealingWarmRestarts', ..., 'polynomial_decay_schedule_with_warmup']Once you've chosen, create the model:#### The optimizer of choice can be passed asfromflash.imageimportImageClassifier# - String valuemodel=ImageClassifier(backbone="resnet18",num_classes=2,optimizer="Adam",lr_scheduler=None)# - Callablemodel=ImageClassifier(backbone="resnet18",num_classes=2,optimizer=functools.partial(torch.optim.Adadelta,eps=0.5),lr_scheduler=None)# - Tuple[string, dict]: (The dict takes in the optimizer kwargs)model=ImageClassifier(backbone="resnet18",num_classes=2,optimizer=("Adadelta",{"epa":0.5}),lr_scheduler=None)#### The scheduler of choice can be passed as a# - String valuemodel=ImageClassifier(backbone="resnet18",num_classes=2,optimizer="Adam",lr_scheduler="constant_schedule")# - Callablemodel=ImageClassifier(backbone="resnet18",num_classes=2,optimizer="Adam",lr_scheduler=functools.partial(CyclicLR,step_size_up=1500,mode='exp_range',gamma=0.5))# - Tuple[string, dict]: (The dict takes in the scheduler kwargs)model=ImageClassifier(backbone="resnet18",num_classes=2,optimizer="Adam",lr_scheduler=("StepLR",{"step_size":10}))You can also register you own custom scheduler recipes beforeahand and use them shown as above:fromflash.imageimportImageClassifier@ImageClassifier.lr_schedulers_registrydefmy_steplr_recipe(optimizer):returntorch.optim.lr_scheduler.StepLR(optimizer,step_size=10)model=ImageClassifier(backbone="resnet18",num_classes=2,optimizer="Adam",lr_scheduler="my_steplr_recipe")Flash TransformsFlash includes some simple augmentations for each task by default, however, you will often want to override these and control your own augmentation recipe.
To this end, Flash supports custom transformations with theInputTransform.
TheInputTransformis like a callback for transforms, with hooks that can be used to apply transforms to samples or batches, on and off the device / accelerator.
In addition, hooks can be specialized to apply transforms only to the input or target.
With these hooks, complex transforms like MixUp can be implemented with ease.
Here's an example (with an albumentations transform thrown in too!):importtorchimportnumpyasnpimportalbumentationsfromflashimportInputTransformfromflash.imageimportImageClassificationDatafromflash.image.classification.input_transformimportAlbumentationsAdapterdefmixup(batch,alpha=1.0):images=batch["input"]targets=batch["target"].float().unsqueeze(1)lam=np.random.beta(alpha,alpha)perm=torch.randperm(images.size(0))batch["input"]=images*lam+images[perm]*(1-lam)batch["target"]=targets*lam+targets[perm]*(1-lam)returnbatchclassMixUpInputTransform(InputTransform):deftrain_input_per_sample_transform(self):returnAlbumentationsAdapter(albumentations.HorizontalFlip(p=0.5))# This will be applied after transferring the batch to the device!deftrain_per_batch_transform_on_device(self):returnmixupdatamodule=ImageClassificationData.from_folders(train_folder="data/train",transform=MixUpInputTransform,batch_size=2,)Flash Zero - PyTorch Recipes from the Command Line!Flash Zero is a zero-code machine learning platform built
directly into lightning-flash
using theLightning CLI.To get started and view the available tasks, run:flash--helpFor example, to train an image classifier for 10 epochs with aresnet50backbone on 2 GPUs using your own data, you can do:flashimage_classification--trainer.max_epochs10--trainer.gpus2--model.backboneresnet50from_folders--train_folder{PATH_TO_DATA}Kaggle Notebook Examples🚢Titanic crash with Lightning⚡Flash🏠House 💵prices predictions with Lightning⚡FlashPlaying 📋tabular with Lightning⚡Flash🙊Toxic comments with Lightning⚡Flash🫁COVID detection with Lightning⚡️FlashContribute!The lightning + Flash team is hard at work building more tasks for common deep-learning use cases. But we're looking for incredible contributors like you to submit new tasks!Join ourSlackand/or read ourCONTRIBUTINGguidelines to get help becoming a contributor!Note:Flash is currently being tested on real-world use cases and is in active development. Pleaseopen an issueif you find anything that isn't working as expected.CommunityFlash is maintained by ourcore contributors.For help or questions, join our huge community onSlack!CitationsWe’re excited to continue the strong legacy of opensource software and have been inspired over the years by Caffe, Theano, Keras, PyTorch, torchbearer, andfast.ai. When/if additional papers are written about this, we’ll be happy to cite these frameworks and the corresponding authors.Flash leverages models from many different frameworks in order to cover such a wide range of domains and tasks. The full list of providers can be found inour documentation.LicensePlease observe the Apache 2.0 license that is listed in this repository.