Updating the parameters of a few nodes in a pre-trained network during training

metro.smiles: <p>Hi,</p>
<p>I am really new to PyTorch and was wondering if there is a way to specify only a subset of neurons (of a particular layer) to update during training and freeze the rest. Say, update only 2500 nodes of the 4096 in AlexNet, FC7. param.requires_grad seems to apply to all the neurons.</p>
<p>Appreciate your inputs.</p>

jhjungCode: <p>I add simple codes. If I understand your question exactly, It will be helpful</p>
<pre><code class="lang-auto">D_parameters = [
    {'params': model.fc1.parameters()},
    {'params': model.fc2.parameters()}
] # define a part of parameters in model

optimizerD = torch.optim.Adam(D_parameters, lr=learning_rate)

for epoch in range(training_epoch):
    ....
    optimizerD.zero_grad() # zero_grad only D_parameters
    # do something 
    loss.backward() # calculate grads of all
    optimizerD.step() # update only D_parameters

</code></pre>

metro.smiles: <p>Hi,</p>
<p>Appreciate your prompt response but this is not exactly what I am looking for. I want to update only a subset of the fc1 parameters, for example. If we update D_parameters, in your example, then all weights from the 4096 nodes of FC1 get updated. What I want is to update a subset of it, say 2900 of them (that I have as a list).</p>

jhjungCode: <p>I think you should reconstruct trained model, and then apply updated a few nodes.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torchvision import models

original_model = models.alexnet(pretrained=True)

class AlexNetConv4(nn.Module):
            def __init__(self):
                super(AlexNetConv4, self).__init__()
                self.features = nn.Sequential(
                    # stop at conv4
                    *list(original_model.features.children())[:-3]
                )
                self.fc1 = nn.Linear(2900, n_outpout, bias = True)
                self.fc2 = nn.Linear(4096-2900, n_outpout, bias = True)

            def forward(self, x):
                x = self.features(x)
                x1 = self.fc1(x[:,:2900])
                x2 = self.fc2(x[:,2900:])
                ....
                return x

model = AlexNetConv4()
</code></pre>

smth: <p>you could use a backward hook on the output on fc2 to zero the gradients going through parts that you want to filter.</p>
<p>For example:</p>
<pre><code class="lang-python">m = nn.Linear(1024, 4096)
input = Variable(torch.randn(128, 1024), requires_grad=True)

out = m(input)
def my_hook(grad):
    grad_clone = grad.clone()
    grad_clone[:, 2500:] = 0
    return grad_clone
h = out.register_hook(my_hook) # zeroes the gradients wrt the outputs for everything that's not 0 to 2500 over all mini-batches

out.backward(grads)
</code></pre>
<p>Edit: edited to incorporate <a class="mention" href="/u/fmassa">@fmassa</a>’s answer below</p>

jhjungCode: <p>Good!! I knew new practical usages of hooking by your answer. Thank you</p>

fmassa: <p>I think it’s better to avoid modifying the gradients in place, but instead return a new gradient, <a href="http://pytorch.org/docs/autograd.html#torch.autograd.Variable.register_hook">as explained in the docs</a>.<br>
So a modified version would be to pass a hook such as:</p>
<pre><code class="lang-python">def my_hook(grad):
    grad_clone = grad.clone()
    grad_clone[:, 2500:] = 0
    return grad_clone
</code></pre>

dlmacedo: <pre><code>m = nn.Linear(1024, 4096)
input = Variable(torch.randn(128, 1024), requires_grad=True)

out = m(input)
h = out.register_hook(my_hook(grad)) 

out.backward(grads)

def my_hook(grad):
    grad_clone = grad.clone()
    grad_clone[:, 2500:] = 0
</code></pre>
<p>Like this?</p>

fmassa: <p>You also need to return <code>grad_clone</code> in your hook</p>

dlmacedo: <p>For sure.</p>
<p>Thanks,</p>
<p>David</p>

Luiza_Sayfullina: <p>Hey,</p>
<p>Just a small nuance question:<br>
<code>out.backward(grads)</code><br>
where from do we take grads variable?  Can we simply write out.backward() ?</p>

WERush: <p>It depends on your loss.<br>
If your loss is a scalar, it is ok to write loss.backward(). If it is a tensor with length &gt; 1, you need to write loss.backward(grads), and the grads has the same size with your out. For example, <code>loss.backward(torch.ones_like(loss.data))</code>.</p>

Rahul_Patel: <p>What if I need the data of Tensor in the <code>hooked_fn</code>. As far as I understand, it will only have access to grads of the Tensor on which it is registered. I would like to modify gradients based on the current data of Tensor. The end goal is to implement Inverting Gradients given in the paper “Deep Reinforcement Learning in Parameterized Action Space”.</p>
<p>EDIT:<br>
We have access to variables defined outside the scope of <code>hooked_fn</code>. Hence, we can simply do <code>data = hooked_tensor.clone().numpy()</code> inside the <code>hooked_fn</code>. Hence <code>new_grad = some_func(data, grad)</code>.</p>