Sparse matrix computation in pytorch

K_awanish: <p>Hi All,</p>
<p>I have a trained and pruned neural network in pytorch. after pruning it, i expected the inference time to be reduced but it doesn’t seem to be, so can someone please suggest me a way to do that?</p>
<p>basically what i want is that the zeroed out weights should not be computed in the forward pass of the graph so that the inference time is reduced.</p>

Soumya_Kundu: <aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/69311857/why-doesnt-torch-pruning-actually-remove-filters-or-weights">
  <header class="source">

      <a href="https://stackoverflow.com/questions/69311857/why-doesnt-torch-pruning-actually-remove-filters-or-weights" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/15670832/anvar-ganiev" target="_blank" rel="noopener nofollow ugc">
    <img alt="Anvar Ganiev" src="https://www.gravatar.com/avatar/f30244c48d39bbc8331af389571d0cd5?s=256&amp;d=identicon&amp;r=PG&amp;f=y&amp;so-version=2" class="thumbnail onebox-avatar" width="256" height="256">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/69311857/why-doesnt-torch-pruning-actually-remove-filters-or-weights" target="_blank" rel="noopener nofollow ugc">Why doesn't torch pruning actually remove filters or weights?</a>
</h4>

<div class="tags">
  <strong>deep-learning, pytorch, pruning</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/15670832/anvar-ganiev" target="_blank" rel="noopener nofollow ugc">
    Anvar Ganiev
  </a>
  on <a href="https://stackoverflow.com/questions/69311857/why-doesnt-torch-pruning-actually-remove-filters-or-weights" target="_blank" rel="noopener nofollow ugc">08:28AM - 24 Sep 21 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>This is a thread to follow for the above.</p>
<p>TLDR:</p>
<blockquote>
<p>As of now, PyTorch prune’s function just works as a weight mask, that’s all. There are no memory savings associated with using <code>torch.nn.utils.prune</code></p>
</blockquote>

K_awanish: <p>thanks <a class="mention" href="/u/soumya_kundu">@Soumya_Kundu</a>, that post is about the same problem i have mentioned here, but it doesn’t contain a clear solution on how I can achieve this in pytorch.</p>

Soumya_Kundu: <p>Here’s a gist explaining this</p>
<blockquote>
<p>a clear solution on how I can achieve this in pytorch</p>
</blockquote>
<aside class="onebox githubgist" data-onebox-src="https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c">
  <header class="source">

      <a href="https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c" target="_blank" rel="noopener nofollow ugc">gist.github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c" target="_blank" rel="noopener nofollow ugc">https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c</a></h4>

  <h5>Pruning_nuances_in_pytorch.py</h5>
  <pre><code class="Python">import torch
import torch.nn as nn
import torch.nn.utils.prune as prune
import numpy as np
import matplotlib.pyplot as plt

class SimpleCNN(nn.Module):
    def __init__(self, conv1_out=32, conv2_out=64):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, conv1_out, 3, 1)</code></pre>
   This file has been truncated. <a href="https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c" target="_blank" rel="noopener nofollow ugc">show original</a>

<p>
</p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>