Package: zennit
ZennitZennit (Zennitexplainsneuralnetworksintorch) is a
high-level framework in Python using Pytorch for explaining/exploring neural
networks. Its design philosophy is intended to provide high customizability and
integration as a standardized solution for applying rule-based attribution
methods in research, with a strong focus on Layerwise Relevance Propagation
(LRP). Zennit strictly requires models to use Pytorch'storch.nn.Modulestructure (including activation functions).Zennit is currently under active development, but should be mostly stable.If you find Zennit useful for your research, please consider citing our relatedpaper:@article{anders2021software,
      author  = {Anders, Christopher J. and
                 Neumann, David and
                 Samek, Wojciech and
                 Müller, Klaus-Robert and
                 Lapuschkin, Sebastian},
      title   = {Software for Dataset-wide XAI: From Local Explanations to Global Insights with {Zennit}, {CoRelAy}, and {ViRelAy}},
      journal = {CoRR},
      volume  = {abs/2106.13200},
      year    = {2021},
}DocumentationThe latest documentation is hosted atzennit.readthedocs.io.InstallTo install directly from PyPI using pip, use:$pipinstallzennitAlternatively, install from a manually cloned repository to try out the examples:$gitclonehttps://github.com/chr5tphr/zennit.git
$pipinstall./zennitUsageAt its heart, Zennit registers hooks at Pytorch's Module level, to modify the
backward pass to produce rule-based attributions like LRP (instead of the usual
gradient). All rules are implemented as hooks
(zennit/rules.py) and most use the LRP basisBasicHook(zennit/core.py).Composites(zennit/composites.py) are a way
of choosing the right hook for the right layer. In addition to the abstractNameMapComposite, which assigns hooks to layers by name, andLayerMapComposite, which assigns hooks to layers based on their Type, there
exist explicitComposites, some of which areEpsilonGammaBox(ZBoxin
input,Epsilonin dense,Gammain convolutions) orEpsilonPlus(Epsilonin dense,ZPlusin convolutions). All composites may be used by directly
importing fromzennit.composites, or by using their snake-case name as key
forzennit.composites.COMPOSITES.Canonizers(zennit/canonizers.py) temporarily
transform models into a canonical form, if required, likeSequentialMergeBatchNorm, which automatically detects and merges BatchNorm
layers followed by linear layers in sequential networks, orAttributeCanonizer, which temporarily overwrites attributes of applicable
modules, e.g. to handle the residual connection in ResNet-Bottleneck modules.Attributors(zennit/attribution.py) directly
execute the necessary steps to apply certain attribution methods, like the
simpleGradient,SmoothGradorOcclusion. An optionalCompositemay
be passed, which will be applied during theAttributor's execution to
compute the modified gradient, or hybrid methods.Using all of these components, an LRP-type attribution for VGG16 with
batch-norm layers with respect to label 0 may be computed using:importtorchfromtorchvision.modelsimportvgg16_bnfromzennit.compositesimportEpsilonGammaBoxfromzennit.canonizersimportSequentialMergeBatchNormfromzennit.attributionimportGradientdata=torch.randn(1,3,224,224)model=vgg16_bn()canonizers=[SequentialMergeBatchNorm()]composite=EpsilonGammaBox(low=-3.,high=3.,canonizers=canonizers)withGradient(model=model,composite=composite)asattributor:out,relevance=attributor(data,torch.eye(1000)[[0]])A similar setup usingthe example scriptproduces the following attribution heatmaps:For more details and examples, have a look at ourdocumentation.More Example HeatmapsMore heatmaps of various attribution methods for VGG16 and ResNet50, all
generated usingshare/example/feed_forward.py, can be found
below.Heatmaps for VGG16Heatmaps for ResNet50ContributingSeeCONTRIBUTING.mdfor detailed instructions on how to contribute.LicenseZennit is licensed under the GNU LESSER GENERAL PUBLIC LICENSE VERSION 3 OR
LATER -- see theLICENSE,COPYINGandCOPYING.LESSERfiles for details.