Package: photosynthesis-metrics
PhotoSynthesis.MetricsPyTorch library with measures and metrics for various image-to-image tasks like denoising, super-resolution,
image generation etc. This easy to use yet flexible and extensive library is developed with focus on reliability
and reproducibility of results. Use your favourite measures as losses for training neural networks with ready-to-use
PyTorch modules.Getting startedimporttorchfromphotosynthesis_metricsimportssimprediction=torch.rand(3,3,256,256)target=torch.rand(3,3,256,256)ssim_index=ssim(prediction,target,data_range=1.)ExamplesStructural Similarity (SSIM)To compute SSIM index as a measure, use lower case function from the library:importtorchfromphotosynthesis_metricsimportssimfromtypingimportUnion,Tupleprediction=torch.rand(3,3,256,256)target=torch.rand(3,3,256,256)ssim_index:Union[torch.Tensor,Tuple[torch.Tensor,torch.Tensor]]=ssim(prediction,target,data_range=1.)In order to use SSIM as a loss function, use corresponding PyTorch module:importtorchfromphotosynthesis_metricsimportSSIMLossloss=SSIMLoss(data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)target=torch.rand(3,3,256,256)output:torch.Tensor=loss(prediction,target)output.backward()Multi-Scale Structural Similarity (MS-SSIM)To compute MS-SSIM index as a measure, use lower case function from the library:importtorchfromphotosynthesis_metricsimportmulti_scale_ssimprediction=torch.rand(3,3,256,256)target=torch.rand(3,3,256,256)ms_ssim_index:torch.Tensor=multi_scale_ssim(prediction,target,data_range=1.)In order to use MS-SSIM as a loss function, use corresponding PyTorch module:importtorchfromphotosynthesis_metricsimportMultiScaleSSIMLossloss=MultiScaleSSIMLoss(data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)target=torch.rand(3,3,256,256)output:torch.Tensor=loss(prediction,target)output.backward()Total Variation (TV)To compute TV as a measure, use lower case function from the library:importtorchfromphotosynthesis_metricsimporttotal_variationdata=torch.rand(3,3,256,256)tv:torch.Tensor=total_variation(data)In order to use TV as a loss function, use corresponding PyTorch module:importtorchfromphotosynthesis_metricsimportTVLossloss=TVLoss()prediction=torch.rand(3,3,256,256,requires_grad=True)output:torch.Tensor=loss(prediction)output.backward()Visual Information Fidelity (VIF)To compute VIF as a measure, use lower case function from the library:importtorchfromphotosynthesis_metricsimportvif_ppredicted=torch.rand(3,3,256,256)target=torch.rand(3,3,256,256)vif:torch.Tensor=vif_p(predicted,target,data_range=1.)In order to use VIF as a loss function, use corresponding PyTorch class:importtorchfromphotosynthesis_metricsimportVIFLossloss=VIFLoss(sigma_n_sq=2.0,data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)target=torch.rand(3,3,256,256)ouput:torch.Tensor=loss(prediction,target)output.backward()Note, that VIFLoss returns1 - VIFvalue.Gradient Magnitude Similarity Deviation (GMSD)This is port of MATLAB version from the authors of original paper.
It can be used both as a measure and as a loss function. In any case it should me minimized.
Usually values of GMSD lie in [0, 0.35] interval.importtorchfromphotosynthesis_metricsimportGMSDLossloss=GMSDLoss(data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)target=torch.rand(3,3,256,256)ouput:torch.Tensor=loss(prediction,target)output.backward()MultiScale GMSD (MS-GMSD)It can be used both as a measure and as a loss function. In any case it should me minimized.
By defualt scale weights are initialized with values from the paper. You can change them by passing a list of 4 variables toscale_weightsargument during initialization. Both GMSD and MS-GMSD computed for greyscale images, but to take contrast changes into account authors propoced to also add chromatic component. Use flagchromaticto use MS-GMSDc version of the lossimporttorchfromphotosynthesis_metricsimportMultiScaleGMSDLossloss=MultiScaleGMSDLoss(chromatic=True,data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)target=torch.rand(3,3,256,256)ouput:torch.Tensor=loss(prediction,target)output.backward()Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE)To computeBRISQUE scoreas a measure, use lower case function from the library:importtorchfromphotosynthesis_metricsimportbrisquefromtypingimportUnion,Tupleprediction=torch.rand(3,3,256,256)brisque_index:torch.Tensor=brisque(prediction,data_range=1.)In order to use BRISQUE as a loss function, use corresponding PyTorch module:importtorchfromphotosynthesis_metricsimportBRISQUELossloss=BRISQUELoss(data_range=1.)prediction=torch.rand(3,3,256,256,requires_grad=True)output:torch.Tensor=loss(prediction)output.backward()Multi-Scale Intrinsic Distance (MSID)UseMSIDclass to computeMSID scorefrom image features,
pre-extracted from some feature extractor network:importtorchfromphotosynthesis_metricsimportMSIDmsid_metric=MSID()prediction_feats=torch.rand(10000,1024)target_feats=torch.rand(10000,1024)msid:torch.Tensor=msid_metric(prediction_feats,target_feats)If image features are not available, extract them using_compute_featsofMSIDclass.
Please note that_compute_featsconsumes a data loader of predefined format.importtorchfromtorch.utils.dataimportDataLoaderfromphotosynthesis_metricsimportMSIDfirst_dl,second_dl=DataLoader(),DataLoader()msid_metric=MSID()first_feats=msid_metric._compute_feats(first_dl)second_feats=msid_metric._compute_feats(second_dl)msid:torch.Tensor=msid_metric(first_feats,second_feats)Frechet Inception Distance(FID)UseFIDclass to computeFID scorefrom image features,
pre-extracted from some feature extractor network:importtorchfromphotosynthesis_metricsimportFIDfid_metric=FID()prediction_feats=torch.rand(10000,1024)target_feats=torch.rand(10000,1024)msid:torch.Tensor=fid_metric(prediction_feats,target_feats)If image features are not available, extract them using_compute_featsofFIDclass.
Please note that_compute_featsconsumes a data loader of predefined format.importtorchfromtorch.utils.dataimportDataLoaderfromphotosynthesis_metricsimportFIDfirst_dl,second_dl=DataLoader(),DataLoader()fid_metric=FID()first_feats=fid_metric._compute_feats(first_dl)second_feats=fid_metric._compute_feats(second_dl)msid:torch.Tensor=fid_metric(first_feats,second_feats)Kernel Inception Distance(KID)UseKIDclass to computeKID scorefrom image features,
pre-extracted from some feature extractor network:importtorchfromphotosynthesis_metricsimportKIDkid_metric=KID()prediction_feats=torch.rand(10000,1024)target_feats=torch.rand(10000,1024)kid:torch.Tensor=kid_metric(prediction_feats,target_feats)If image features are not available, extract them using_compute_featsofKIDclass.
Please note that_compute_featsconsumes a data loader of predefined format.importtorchfromtorch.utils.dataimportDataLoaderfromphotosynthesis_metricsimportKIDfirst_dl,second_dl=DataLoader(),DataLoader()kid_metric=KID()first_feats=kid_metric._compute_feats(first_dl)second_feats=kid_metric._compute_feats(second_dl)kid:torch.Tensor=kid_metric(first_feats,second_feats)Geometry Score (GS)UseGSclass to computeGeometry Scorefrom image features,
pre-extracted from some feature extractor network. Computation is heavily CPU dependent, adjustnum_workersparameter according to your system configuration:importtorchfromphotosynthesis_metricsimportGSgs_metric=GS(sample_size=64,num_iters=100,i_max=100,num_workers=4)prediction_feats=torch.rand(10000,1024)target_feats=torch.rand(10000,1024)gs:torch.Tensor=gs_metric(prediction_feats,target_feats)GS metric requiersgudhilibrary which is not installed by default.
If you use conda, write:conda install -c conda-forge gudhi, otherwise followinstallation guide.Inception Score(IS)Useinception_scorefunction to computeISfrom image features,
pre-extracted from some feature extractor network. Note, that we follow recomendations from paperA Note on the Inception Score, which proposed small modification to original algorithm:importtorchfromphotosynthesis_metricsimportinception_scoreprediction_feats=torch.rand(10000,1024)mean:torch.Tensor,variance:torch.Tensor=inception_score(prediction_feats,num_splits=10)To compute difference between IS for 2 sets of image features, useISclass.importtorchfromphotosynthesis_metricsimportISis_metric=IS(distance='l1')prediction_feats=torch.rand(10000,1024)target_feats=torch.rand(10000,1024)distance:torch.Tensor=is_metric(prediction_feats,target_feats)Table of ContentsOverviewInstallationRoadmapCommunityContributingContactAcknowledgementsOverviewPhotoSynthesis.Metricshelps you to concentrate on your experiments without the boilerplate code.
The library contains a set of measures and metrics that is constantly getting extended.
For measures/metrics that can be used as loss functions, corresponding PyTorch modules are implemented.Installation$ pip install photosynthesis-metricsIf you want to use the latest features straight from the master, clone the repo:$gitclonehttps://github.com/photosynthesis-team/photosynthesis.metrics.gitRoadmapSee theopen issuesfor a list of proposed
features and known issues.CommunityContributingWe appreciate all contributions. If you plan to:contribute back bug-fixes, please do so without any further discussionclose one ofopen issues, please do so if no one has been assigned to itcontribute new features, utility functions or extensions, please first open an issue and discuss the feature with usPlease see thecontribution guidefor more information.ContactSergey Kastryulin-@snk4tr-snk4tr@gmail.comProject Link:https://github.com/photosynthesis-team/photosynthesis.metricsPhotoSynthesis Team:https://github.com/photosynthesis-teamOther projects by PhotoSynthesis Team:PhotoSynthesis.ModelsAcknowledgementsPavel Parunin-@PavelParunin- idea proposal and developmentDjamil Zakirov-@zakajd- developmentDenis Prokopenko-@denproc- development