CNN Filter pruning methods in PyTorch

TonightHitTiger1: <p><strong>Hi, I wanna implement network pruning using PyTorch.</strong><br>
<strong>First, I use pruning algorithm to prune the model.</strong><br>
<strong>the result as follow:</strong><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/9/9efba291b57fee4a17e7b0e403415dfcaeeaad93.png" data-download-href="https://discuss.pytorch.org/uploads/default/9efba291b57fee4a17e7b0e403415dfcaeeaad93" title="a"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/9/9efba291b57fee4a17e7b0e403415dfcaeeaad93_2_690x407.png" alt="a" data-base62-sha1="mGqz5lUUq3xNbFss0gdY4mOOExB" width="690" height="407" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/9/9efba291b57fee4a17e7b0e403415dfcaeeaad93_2_690x407.png, https://discuss.pytorch.org/uploads/default/original/2X/9/9efba291b57fee4a17e7b0e403415dfcaeeaad93.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/9/9efba291b57fee4a17e7b0e403415dfcaeeaad93.png 2x" data-dominant-color="F0F0F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">a</span><span class="informations">886×523 59.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<strong>then retrain the model.</strong><br>
<strong>but These cut weights are trained to be non-zero.</strong><br>
<strong>I wanna freeze only zero weights in entire network.</strong><br>
<strong>In this context, freeze means that freezed weights cannot be trained anymore.</strong><br>
<strong>The following method is not feasible</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/3/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2.png" data-download-href="https://discuss.pytorch.org/uploads/default/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2" title="b"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/3/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2_2_690x212.png" alt="b" data-base62-sha1="7rBKlxkUiqhqCB9VMZbdLR6AgUO" width="690" height="212" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/3/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2_2_690x212.png, https://discuss.pytorch.org/uploads/default/optimized/2X/3/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2_2_1035x318.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/3/342e2b8e7d1d0869f2fd40a3c71444b89b3cf4e2_2_1380x424.png 2x" data-dominant-color="F3F1F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">b</span><span class="informations">1551×477 90.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<strong>Is there a better solution?</strong></p>

ptrblck: <p>It looks like you would like to set the <code>requires_grad</code> flag to <code>False</code> for a specific kernel of your weight matrix.<br>
This is not possible as you can only set <code>requires_grad</code> for the whole matrix.<br>
However, you could zero out the gradients for this kernel after <code>backward</code> and before the optimizer is called.</p>

sarvani: <p>Hi ptrblck…can you please suggest which version of pytorch pruning will modify the architecture of the network,when filters are pruned?<br>
I am currently using " nn.utils.prune" and  is not removing the filters from original architecture.</p>

ptrblck: <p>Unfortunately, I’m not familiar enough with the pruning utility, but <a class="mention" href="/u/michela">@Michela</a> might know. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

sarvani: <p>thanks for the reply.<br>
Hoping for a suggestion from Michela.</p>