Package: interpret-pytorch
PyTorch InterpretA simple to use PyTorch library for interpreting your deep learning results, using both visualisations and attributions. Inspired byTensorFlow Lucid.InstallationInstall from PyPI:pipinstallinterpret-pytorchOr, install the latest code from GitHub:pipinstallgit+https://github.com/ttumiel/interpretDependenciesinterpretrequires a working installation of PyTorch.ContentsTutorialsVisualisationQuickstartParameterisationsObjectivesAttributionQuickstartMiscellaneous InterpretationsPlot Top LossesPlot Confusion MatrixPlot Dataset ExamplesTutorialsRun the tutorials in the browser using Google Colab.TutorialLinkIntroduction tointerpretVisualisation TutorialMiscellaneous Methods TutorialVisualisationVisualisation is a technique that generates inputs that optimise a particular objective within a trained network. By using visualisations, we can understand what it is that a network is looking for. For an in-depth explanation of visualisation, seeFeature Visualisation.QuickstartGenerating visualisations is done by loading a trained network, selecting the objective to optimise for and running the optimisation. An example using a pretrained network fromtorchvisionis shown.frominterpretimportOptVisimporttorchvision# Get the PyTorch neural networknetwork=torchvision.models.vgg11(pretrained=True)# Select a layer from the network. Use get_layer_names()# to see a list of layer names and sizes.layer='features/18'channel=12# Create an OptVis object from a PyTorch modeloptvis=OptVis.from_layer(network,layer=layer,channel=channel)# Create visualisationoptvis.vis()ParameterisationsImages can be parameterised in several different ways. As long as the parameterisation is differentiable, the input can be optimised for a particular layer. For code examples, see theVisualisation Tutorial Notebook.The default parameterisation is in spatial and colour decorrelated space.We can also parameterise in regular pixel space but the visualisations tend to be worse.Another parameterisation is a compositional pattern producing network (CPPN) which can generate infinite resolution images that have the effect of "light paintings."ObjectivesThe objective on which to optimise can also be manipulated to create different visualisations. We can add objectives together to get compound objectives or negate them to get negative neurons. See theVisualisation Tutorial Notebookfor examples.Layer ObjectiveA LayerObjective can be created easily using thefrom_layerOptVis class method. In this function, we can choose the layer, channel and neuron to optimise for. Here we can optimise for a particular neuron:We can also manually create two objectives and add them together to get a compound objective:Or we can find the negated objective that minimises a particular neuron:Layer objectives are fairly flexible. You can select any layer in the network and capture the output of that particular layer. We can visualise the last layer of the network, generating class visualisations of the different classes in ImageNet.Deep Dream ObjectiveThe deep dream objective optimises for "interestingness" across an entire layer. We can create this objective from an input image and select a layer using thefrom_dreamclass method.AttributionNetwork attribution is done by feeding a particular input into the trained network and generating a saliency map that shows the parts of the image that the network activates highly on.QuickstartfrominterpretimportGradcam,normfromPILimportImageimporttorchvisionnetwork=torchvision.models.vgg11(pretrained=True)input_img=Image.open('image.jpg')# Normalise the input image and turn it into a tensorinput_data=norm(input_img)# Select the class that we are attributing toclass_number=207# Choose a layer for Grad-CAMlayer='features/20'# Generate a Grad-CAM attribution mapsaliency_map=Gradcam(network,input_data,im_class=class_number,layer=layer)saliency_map.show()Miscellaneous InterpretationsIncluded ininterpretare a few additional interpretation methods that don't neatly fit into visualisation or attribution methods.Plot Top LossesPlot the inputs that result in the largest loss. Useful for identifying where your network is most unsure or where the inputs actually don't fit the label given (a mislabelled image). You can also enable a Grad-CAM attribution overlay for each image so that you can tell where the network is looking.Plot Confusion MatrixPlot a confusion matrix for a multi-class classification or binned regression objective.Plot Dataset ExamplesPlot some dataset examples that maximise a particularLayerObjectivefrom the visualisation objectives described above. Useful for identifying clear examples of what the network is looking for in a particular visualisation using real examples.