No module named 'torch.nn.utils.prune'

mehran2020: <p>Hello everyone</p>
<p>I have been trying to use the pruning module in PyTorch. However, when I want to lead the module:</p>
<pre><code class="lang-auto">import torch.nn.utils.prune
</code></pre>
<p>I get the error:<br>
ModuleNotFoundError: No module named ‘torch.nn.utils.prune’</p>
<p>Has Pytorch replaced the keyword or something?</p>
<p>Thanks</p>

ptrblck: <p>Could you check your current PyTorch version? If I’m not mistaken, <code>torch.nn.utils.prune</code> was introduced in <code>1.4.0</code>, so you would need this or a newer version.</p>

mehran2020: <p>Thanks for the reply. I have torch 1.3.1!</p>

ani0075: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>All the inbuilt methods for the prune module are based on norm of parameters (weights and biases). Any suggestion if I want to prune connections based on activations (e.g. remove top 10% connections based on norm of activations)? Should I try implementing it by subclassing or do you know if it has been implemented someplace else?</p>
<p>Also is it easy to switch the ranking of torch.nn.utils.prune.l1_unstructured? It says “removing the specified amount of (currently unpruned) units with the lowest L1-norm”. What if I want to remove the ones with the highest L1-norm? Let’s assume I have an application for the opposite ranking. Not sure if it makes sense according to pruning literature,</p>

ptrblck: <p>I haven’t implemented custom pruning methods so far.<br>
However, <a class="mention" href="/u/michela">@Michela</a> presents the pruning module as well as ways to customize it in <a href="https://www.youtube.com/watch?v=TaOwEa3m5dw">this talk</a>, which might be a good starter. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Michela: <p><a class="mention" href="/u/ani0075">@ani0075</a> You can extend the module with whatever pruning technique you want. There is an ongoing conversation about what to include here at this issue: <a href="https://github.com/pytorch/pytorch/issues/39765" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/39765</a></p>
<p>I can help with the implementation, whether it ends up making it into the official list of pytorch supported methods, or it ends up living in your own separate repository.<br>
Either way, I’d recommend subclassing <code>BasePruningMethod</code> so that you can take advantage of all the pre-packaged functionalities provided by the base class, and remain compatible with the pruning “language” of PyTorch pruning. The <a href="https://pytorch.org/tutorials/intermediate/pruning_tutorial.html#extending-torch-nn-utils-prune-with-custom-pruning-functions" rel="nofollow noopener">bottom portion of the pruning tutorial</a> shows how to extend the module with your own custom pruning module (through a very simple example, in that case).</p>
<p>For reverse L1 unstructured, this should be a super easy change in the <code>LnStructured</code> code from <code>largest=True</code> to <code>largest=False</code> when computing the <code>topk</code>, as documented in the code here: <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/prune.py#L709" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/prune.py#L709</a></p>

sarvani: <p>Dear <a class="mention" href="/u/michela">@Michela</a> ,</p>
<p>I am currently using “nn.utils.prune” following your colab tutorial.I could only zero the weight but not remove.</p>
<p>Is there any specific method in “nn.utils.prune” to modify architecture?</p>