Pytorch Pruning not working

Raghav_Gurbaxani: <p>Hi all,</p>
<p>I am trying to prune my pytroch model based on the tutorial [here]. (<a href="https://pytorch.org/tutorials/intermediate/pruning_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/intermediate/pruning_tutorial.html</a>)</p>
<p>However, the size of the model doesn’t reduce (even with 40 % pruning).</p>
<pre><code class="lang-auto">Original Size:
Size (MB): 6.623636
Pruned model size
Size (MB): 6.623636
</code></pre>
<p>The link to my code can be found here -<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py" target="_blank" rel="nofollow noopener">raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py</a></h4>
<pre><code class="lang-py">import os
import config as cfg
from model import East
import torch
import utils
import preprossing
import cv2
import numpy as np
import time
import loss

def uninplace(model):
    if hasattr(model, 'inplace'):
        model.inplace = False
    if not model.children():
        return
    for child in model.children():
        uninplace(child)
        
def print_size_of_model(model):
</code></pre>

  This file has been truncated. <a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>moreover when I prune the bias as well I get the following error -</p>
<pre><code class="lang-auto">    import torch.nn.utils.prune as prune
    for name, module in model.named_modules():
    # prune 40% of connections in all 2D-conv layers
        if isinstance(module, torch.nn.Conv2d):
            prune.l1_unstructured(module, name='weight', amount=0.4)
            prune.l1_unstructured(module, name='bias', amount=0.3)
            prune.remove(module, 'weight')
            prune.remove(module, 'bias')
    # prune 40% of connections in all linear layers
        elif isinstance(module, torch.nn.Linear):
            prune.l1_unstructured(module, name='weight', amount=0.4)
            prune.l1_unstructured(module, name='bias', amount=0.4)
            prune.remove(module, 'weight')
            prune.remove(module, 'bias')
</code></pre>
<p>ERROR -</p>
<pre><code class="lang-auto">
  File "/Users/raghavgurbaxani/Desktop/DI PA /EAST-Pytorch/try_pruning.py", line 143, in main
    prune.l1_unstructured(module, name='bias', amount=0.3)

  File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/utils/prune.py", line 886, in l1_unstructured
    L1Unstructured.apply(module, name, amount)

  File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/utils/prune.py", line 536, in apply
    return super(L1Unstructured, cls).apply(module, name, amount=amount)

  File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/utils/prune.py", line 167, in apply
    default_mask = torch.ones_like(orig)  # temp

TypeError: ones_like(): argument 'input' (position 1) must be Tensor, not NoneType
</code></pre>
<p><a class="mention" href="/u/michela">@Michela</a> could you help out on whats going wrong here  ? <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:"> <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:"></p>

Michela: <p>The model size is not expected to change unless you turn your tensors in coordinate representation using <code>to_sparse()</code>. This will only give you advantages at higher levels of sparsity though. Otherwise, all you’re doing is having the same exact tensors as before, now with a bunch of values set to zero instead of whatever number they used to be, so the size will stay the same.</p>
<p>I’ll look into your error now.</p>

Michela: <p>Do you have a minimum example that runs that reproduces this error?</p>

Raghav_Gurbaxani: <p>Hi <a class="mention" href="/u/michela">@Michela</a></p>
<p>you can try running my script <a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py" rel="nofollow noopener">https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/try_pruning.py</a></p>
<p>based on this repository -<br>
</p><aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">
      <a href="https://github.com/ZhangMingliangAI/EAST-Pytorch" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/38106776?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/ZhangMingliangAI/EAST-Pytorch" target="_blank" rel="nofollow noopener">ZhangMingliangAI/EAST-Pytorch</a></h3>

<p>This is a project of EAST text detector implemented by Pytorch - ZhangMingliangAI/EAST-Pytorch</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>pruning only the weight works, but as soon as I prune the bias as well it throws me the error</p>

Michela: <p>What is <code>utils</code> here?</p>

Raghav_Gurbaxani: <p>here’s the script for utils -<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/utils.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/utils.py" target="_blank" rel="nofollow noopener">raghavgurbaxani/Quantization_Experiments/blob/master/utils.py</a></h4>
<pre><code class="lang-py">import torch
import os
import config as cfg
from torch.nn import init
import cv2
import numpy as np
import time
import preprossing
import locality_aware_nms


def init_weights(m_list, init_type=cfg.init_type, gain=0.02):
    print("EAST &lt;==&gt; Prepare &lt;==&gt; Init Network'{}' &lt;==&gt; Begin".format(cfg.init_type))
    # this will apply to each layer
    for m in m_list:
        classname = m.__class__.__name__
        if hasattr(m, 'weight') and (classname.find('Conv') != -1 or classname.find('Linear') != -1):
            if init_type == 'normal':
                init.normal_(m.weight.data, 0.0, gain)
            elif init_type == 'xavier':
</code></pre>

  This file has been truncated. <a href="https://github.com/raghavgurbaxani/Quantization_Experiments/blob/master/utils.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Also can you suggest the best pruning method for optimal accuracy vs inference speed ? As the model size reduces, atleast can I expect a faster inference speed ?</p>

Raghav_Gurbaxani: <p>also <a class="mention" href="/u/michela">@Michela</a></p>
<p>sorry the code is based on this repo<br>
</p><aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/ZhangMingliangAI/EAST-Pytorch" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/38106776?s=400&amp;v=4" class="thumbnail onebox-avatar" width="128" height="128">

<h3><a href="https://github.com/ZhangMingliangAI/EAST-Pytorch" target="_blank" rel="nofollow noopener">ZhangMingliangAI/EAST-Pytorch</a></h3>

<p>This is a project of EAST text detector implemented by Pytorch - ZhangMingliangAI/EAST-Pytorch</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>my bad</p>

Michela: <p>What’s <code>preprossing</code>? I’m sorry but I can’t debug your entire repository. If you have a simple script that reproduces the error, I can look into it.</p>
<p>There is no optimal method for accuracy and/or speed. Also, you cannot expect faster inference speed using pytorch pruning at the moment. The feature is experimental and not powered by a fast sparse linear algebra library yet.</p>

Raghav_Gurbaxani: <p><a class="mention" href="/u/michela">@Michela</a> thank you for your support, sorry I pasted the wrong repository (corrected it now).</p>
<p>If you just run my code within that repo it should reproduce the error . I appreciate your help on this.</p>
<p>Is there any plan on more mature pruning releases in the future ?</p>

Raghav_Gurbaxani: <p><a class="mention" href="/u/michela">@Michela</a> any update on this ?</p>