Package: vinn
VINNVINN is Visual Interpretation of Neural NetworksVINN includes a series of interpretability algorithms, like LIME, Grad CAM, Integrated Gradient, saliency maps and so on.VINN is a graduation project, and will update to new version if possible.Target audienceDevelopers, who are looking to understand what their deep models have learnt, or researchers, who want to test their new algorithms are the potential audiences for VINN.VINN standardize the algorithm framework, so that you can follow the framework to construct your own algorithm and test it.Installationrequirementsyou can create a new environment bycondacreate-frequirementsor make sure your own environments havepython >= 3.6pytorch >= 1.2installing the latest releasepip:pipinstallvinnconda:condainstallvinnor you can download from GitHub and install in the repositoryhttps://github.com/Torato-Taraka/VINN.git
pipinstallvinn-1.0-py3-none-any.whlGetting startimporttorchimporttorchvision.transformsastfsfromtorchvision.modelsimportresnet18importcv2model=resnet18(pretrained=True)transform=tfs.Compose([tfs.ToTensor(),tfs.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])last_conv=model.layer4[-1]image_path='test.jpg'image=cv2.imread(image_path)image=cv2.resize(image,(256,256))image=cv2.cvtColor(image,cv2.COLOR_BGR2RGB)gpu=TrueTill now, necessary parameters(model, image, transform, gpu) is ready, and you can use VINN to interpret your modelimportvinngradcam=vinn.GradCAM(model=model,image=image,last_conv=last_conv,transform=transform,gpu=gpu)gradcam.forward()After forward, the algorithm finish processing, and the result is stored ingradcam.mask.You can visualize the result like the following:params={"original image",gradcam.image,"GradCAM",gradcam.mask,"masked image",gradcam.masked_image}vinn.subplot([params])Note that you should pass a list of dicts, since that you may need to interpret several images at the same time.And you can see how the image masked by callinggradcam.masked_image.But pay attention that NOT all the algorithms generates mask and original image together, only GradCAM, GuidedGradCAM, LIME and Sensitivity have masked_image.Of course you can generate by yourself and maybe the later version will achieve the function.Customize models, datasets and algorithmsVINN support customizing your model, datasets and even algorithmsmodelsYou should organize your model likefromvinn.modelimportModels@Models("yourmodel")defyourmodel():classyourmodel(nn.Module):def__init__(self):...returnyourmodel()This is for registering your algorithm in order to managementdatasetsSimilar to customize modelsfromvinn.datasetsimportDataSet@DataSet("yourdataset")defyourdataset():classyourdataset:def__init__(self):...def__getitem__(self,index):image=self.transform(self.images[index])lable=self.lables[index]returnimage,labledef__len__(self):returnlen(self.labels)returnyourdataset()__getitem__()and__len__()is necessary, you can see the details at [torch.utils.data.Dataset](torch.utils.data — PyTorch 1.10 documentation)And you can see all the existing dataset byfromvinnimportlist_datasetlist_dataset()train modelfromvinnimporttrainerfromtorchvisionimporttransformsastfstransform=tfs.Compose([...])trainer(model_name='yourmodel',dataset_name='yourdataset',classes=...,transform=transform,train_size=...,test_size=...,lr=...,epochs=...,gpu=True)model_name: the string in the @Models() you definedataset_name: the string in the @Dataset() you defineclasses: the classes of your datasettransform: you define ittrain_size: the size of a sample from your train datatest_size: the size of a sample from your test datalr: learning rateepochs: training timesgpu: whether use cuda or notalgorithmsYou should first organize your algorithm likefromvinn.algorithmimportAlgorithmclassyouralgo(Algorithm):def__init__(self,model,image,transform,gpu,...):super().__init__(model,image,transform,gpu)...defforward(self):...self.mask=maskAlgorithmis the father class of all algorithmsThe most important thing is that you must pass the result toself.mask, so that you can easily visualize your result.Or you can return your result, but I don't think it good for your following operation.References of algorithmsIntegratedGradients:Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017SmoothGrad:SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017SaliencyMap:Deep Inside Convolutional Networks: Visualising
Image Classification Models and Saliency Maps, K. Simonyan, et. al. 2014GradCAM,GuidedGradCAM:Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017LIME:"Why Should I Trust You?": Explaining the Predictions of Any ClassifierGBP(Guided Backpropagation):Striving for Simplicity: The All Convolutional Net, Jost Tobias Springenberg et al. 2015Sensitivity:Interpretable Explanations of Black Boxes by Meaningful PerturbationLicenseVINN is MIT licensed, as found in LICENSE file.