Layer-wise Relevance Propagation (LRP) in PyTorch

username: vdw
cooked: <p>I get the intuition behind <a href="http://www.heatmapping.org/" rel="nofollow noopener">LRP</a> and I would like to implement in PyTorch. However, I’m not that familiar with the PyTorch internals to know how to best get started.</p>
<p>The website links to a LRP wrapper for Tensorflow. When looking at the code, it seems one has to wrap each layer to add some LRP specific code snippet. I assume the same would be required for PyTorch.</p>
<p>Does anyone already implemented LRP in Pytorch? Or does anyone has some opinions how to tackle this?</p>

username: tom
cooked: <p>So what I did was to implement <code>autograd.Functions</code> that use the regular forward but implement the LRP rules for backward. Then I mirrored all <code>nn.Modules</code> my network uses. Worked for LSTMs (ULMFiT) and ResNets and some others for me.<br>
Note that “unconstrained” LRP for ResNets does have continuity issues with the residual connections, <a href="https://github.com/albermax/innvestigate/issues/2" rel="nofollow noopener">here is a discussion involving some coauthors of some papers</a>.</p>
<p>Best regards</p>
<p>Thomas</p>

username: vdw
cooked: <p><a class="mention" href="/u/tom">@tom</a> any chance you can share some example code for a module/layer? I’m still getting more into the details of PyTorch.</p>

username: dmitrysarov
cooked: <p>I had worked on this task. Here some results, not much but can considered as starting point.<br>
<a href="https://github.com/dmitrysarov/LRP_decomposition/blob/master/LRP_notebook.ipynb" class="onebox" target="_blank" rel="noopener nofollow ugc">https://github.com/dmitrysarov/LRP_decomposition/blob/master/LRP_notebook.ipynb</a><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/2/262bc2bf9a68689ce03a6da4052726e1a4f71090.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/262bc2bf9a68689ce03a6da4052726e1a4f71090" title="work"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/2/262bc2bf9a68689ce03a6da4052726e1a4f71090_2_690x388.jpeg" alt="work" data-base62-sha1="5rFTQL3XplAXIugulaf6wiiJ5kI" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/2/262bc2bf9a68689ce03a6da4052726e1a4f71090_2_690x388.jpeg, https://discuss.pytorch.org/uploads/default/original/2X/2/262bc2bf9a68689ce03a6da4052726e1a4f71090.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/2/262bc2bf9a68689ce03a6da4052726e1a4f71090.jpeg 2x" data-dominant-color="7F8B73"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">work</span><span class="informations">800×450 80.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: skynet10100
cooked: <p><a class="mention" href="/u/vdw">@vdw</a> did you get to managed to implement LRP for PyTorch?<br>
Best Regrads</p>

username: skynet10100
cooked: <p>If you are still looking for some explanation see <a href="http://heatmapping.org/tutorial/" rel="nofollow noopener">heatmapping</a> Seems to be a good starting point.</p>