Package: blowtorch
blowtorchIntuitive, high-level training framework for research and development. Abstracts away boilerplate normally associated with training and evaluating PyTorch models, without limiting flexibility. Blowtorch provides the following:A way to specify training runs at a high level, while not giving up on fine-grained control over individual training partsAutomated checkpointing, logging and resuming of runsAsacredinspired configuration managementReproducibility by keeping track of configuration, code and random state of each runInstallationMake sure you havenumpyandtorchinstalled, then install with pip:pipinstall--upgradeblowtorchExamplefromtorch.optimimportAdamfromtorch.utils.dataimportDataLoaderfromtorchvision.datasetsimportCIFAR10fromtorchvision.transformsimportToTensorfromtorchvision.modelsimportvgg16fromblowtorchimportRunrun=Run(random_seed=123)@run.train_step@run.validate_stepdefstep(batch,model):x,y=batchy_hat=model(x)loss=(y-y_hat).square().mean()returnloss# will be called when model has been moved to the desired device@run.configure_optimizersdefconfigure_optimizers(model):returnAdam(model.parameters())train_loader=DataLoader(CIFAR10('.',train=True,download=True,transform=ToTensor()))val_loader=DataLoader(CIFAR10('.',train=False,download=True,transform=ToTensor()))run(vgg16(num_classes=10),train_loader,val_loader)ConfigurationYou can pass multiple configuration files in YAML format to yourRun, e.g.run=Run(config_files=['config/default.yaml'])Configuration values can then be accessed via e.g.run['model']['num_layers']. Dotted notation is also supported, e.g.run['model.num_layers'].  When executing your training script, individual configuration values can be overwritten as follows:pythontrain.pywithmodel.num_layers=4model.use_dropout=TrueRun optionsRun.run()takes following options:model:torch.nn.Moduletrain_loader:torch.utils.data.DataLoaderval_loader:torch.utils.data.DataLoaderloggers:Optional[List[aurora.logging.BaseLogger]](List of loggers that subscribe to various logging events, see logging section)max_epochs:int(default1)use_gpu:bool(defaultTrue)gpu_id:int(default0)resume_checkpoint:Optional[Union[str, pathlib.Path]](Path to checkpoint directory to resume training from, defaultNone)save_path:Union[str, pathlib.Path](Path to directory that blowtorch will save logs and checkpoints to, default'train_logs')run_name:Optional[str](Name associated with that run, will be randomly created if None, defaultNone)optimize_metric:Optional[str](train metric that will be used for optimization, will pick the first returned one if None, defaultNone)checkpoint_metric:Optional[str](validation metric that will be used for checkpointing, will pick the first returned one if None, defaultNone)smaller_is_better:bool(defaultTrue)optimize_first:bool(whether optimization should occur during the first epoch, defaultFalse)detect_anomalies:bool(enable autograd anomaly detection, defaultFalse)LoggingBlowtorch will create a folder with name "[timestamp]-[name]-[sequential integer]" for each run inside therun.save_pathdirectory. Here it will save the runs's configuration, metrics, a model summary, checkoints as well as source code. Additional loggers can be added throughRunsloggersparameter:blowtorch.loggers.WandbLogger: Logs to Weights & Biasesblowtorch.loggers.TensorBoardLogger: Logs to TensorBoardCustom loggers can be created by subclassingblowtorch.loggers.BaseLogger.DecoratorsBlowtorch uses the decorator syntax to specify parts of the training pipeline:@run.train_step,@run.val_step: Specify train/val steps with one or two functions. Arguments:batch,model,is_validate,device,epoch@run.train_epoch,@run.val_epoch: Specify whole train/val epoch, in case more flexibility for iteration/optimization is required. Arguments:data_loader,model,is_validate,optimizers@run.configure_optimizers: Return optimizers and learning rate schedulers. Can either return a single optimizer object or a dictionary with multiple optimizers/schedulers. Arguments:model