Package: homura-core
homuramasterdevhomurais a fast prototyping library for DL research.🔥🔥🔥🔥homura(焰) isflameorblazein Japanese. 🔥🔥🔥🔥RequirementsMinimal requirementsPython>=3.9
PyTorch>=1.9.0
torchvision>=0.10.0Installationpip install -U homura-coreorpip uninstall homurapip install -U git+https://github.com/moskomule/homuraOptionalfaiss (for faster kNN)
accimage (for faster image pre-processing)
cupy (homura)
opt_einsum (for accelerated einsum)testpytest .APIsBasicshomuraaims abstract (e.g., device-agnostic) simple prototyping.fromhomuraimportoptim,lr_schedulerfromhomuraimporttrainers,reportersfromhomura.visionimportMODEL_REGISTRY,DATASET_REGISTRYfromtorch.nnimportfunctionalasFtrain_loader,test_loader,num_classes=DATASET_REGISTRY('dataset_name')(...)# User does not need to care about the devicemodel=MODEL_REGISTRY('model_name')(num_classes=num_classes)# Model is registered in optimizer lazily. This is convenient for distributed training and other complicated scenes.optimizer=optim.SGD(lr=0.1,momentum=0.9)scheduler=lr_scheduler.MultiStepLR(milestones=[30,80],gamma=0.1)withtrainers.SupervisedTrainer(model,optimizer,F.cross_entropy,reporters=[reporters.TensorboardReporter(...)],scheduler=scheduler)astrainer:# epoch-based trainingfor_intrainer.epoch_iterator(num_epochs):trainer.train(train_loader)trainer.scheduler.step()trainer.test(test_loader)trainer.scheduler.step()# otherwise, iteration-based trainingtrainer.run(train_loader,test_loader,total_iterations=1_000,val_intervals=10)print(f"Max Accuracy={max(trainer.history['accuracy']['tests'])}")You can customizeiterationoftraineras follows.fromhomura.trainersimportTrainerBase,SupervisedTrainerfromhomura.metricsimportaccuracytrainer=SupervisedTrainer(...)# from v2020.08, iteration is much simplerdefiteration(trainer:TrainerBase,data:Tuple[torch.Tensor,torch.Tensor])->None:input,target=dataoutput=trainer.model(input)loss=trainer.loss_f(output,target)trainer.reporter.add('loss',loss.detach())trainer.reporter.add('accuracy',accuracy(input,target))trainer.reporter.add('')iftrainer.is_train:trainer.optimizer.zero_grad()loss.backward()trainer.optimizer.step()# in case schedule is step-wisetrainer.scheduler.step()SupervisedTrainer.iteration=iteration# ortrainer.update_iteration(iteration)dictof models, optimizers, loss functions are supported. This is useful for GANs, for example.trainer=CustomTrainer({"generator":generator,"discriminator":discriminator},{"generator":gen_opt,"discriminator":dis_opt},{"reconstruction":recon_loss,"generator":gen_loss},**kwargs)reporterinternally tracks the values during each epoch and reduces after every epoch. Therefore, users can compute
mIoU, for example, asfromhomura.metricsimportconfusion_matrixdefcm_to_miou(cms:List[torch.Tensor])->torch.Tensor:# cms: list of confusion matricescm=sum(cms).float()miou=cm.diag()/(cm.sum(0)+cm.sum(1)-cm.diag())returnmiou.mean().item()defiteration(trainer:TrainerBase,data:Tuple[torch.Tensor,torch.Tensor])->None:input,target=dataoutput=trainer.model(input)trainer.reporter.add('miou',confusion_matrix(output,target),reduction=cm_to_miou)...Distributed trainingDistributed training is complicated at glance.homurahas simple APIs, to hide the messy codes for DDP, such
ashomura.init_distributedfor the initialization andhomura.is_masterfor checking if the process is master or not.For details, seeexamples/imagenet.py.ReproducibilityThese methods make randomness deterministic in its context.fromhomura.utils.reproducibilityimportset_deterministic,set_seedwithset_deterministic(seed):# suppress nondeterministic computation# but will affect the performancesomething()withset_seed(seed):# only set random seed of Python, PyTorch and Numpyother_thing()Registry SystemFollowing major libraries,homuraalso has a simple registry system.fromhomuraimportRegistryMODEL_REGISTRY=Registry("language_models")@MODEL_REGISTRY.registerclassTransformer(nn.Module):...# orMODEL_REGISTRY.register(bert_model,'bert')# magicMODEL_REGISTRY.import_modules(".")transformer=MODEL_REGISTRY('Transformer')(...)# orbert=MODEL_REGISTRY('bert',...)ExamplesSeeexamples.Citing@misc{homura,author={Ryuichiro Hataya},title={homura},year={2018},howpublished={\url{https:/github.com/moskomule/homura}},}