Why is it when I call require_grad = False on all my params my weights in the network would still update?

inkplay: <p>What I am trying to do right now is to write a multi layer conv2d encoder and freeze the weights from updating for the earlier layers. This hopefully would give me back a similar effect like progressively growing the layers. This way I can initialize the complete network first without worrying about how to mix and match and add new layers to the network. So before I start to write a complex model I thought I would experiment with freezing a small network for testing purpose. The result is already different from what I was expecting. I assumed setting grad to zero would stop the updating of the weights so the end weight would stay the same but I was wrong. Below is my testing code.</p>
<pre><code class="lang-auto">encoder = nn.Sequential( nn.Conv2d(1, 4, 1), nn.ReLU() ).cuda()

criterion = torch.nn.BCELoss()
optimizer = torch.optim.Adam(encoder.parameters(), lr=0.001)

for params in encoder.parameters():
    print('params:' params)
    prams.require_grad = False # Freeze all weights

params: [ [[[ 0.1500]]], [[[0.9332]]], [[[-0.1422]]], [[[-0.7685]] ] ....

epochs = 2
target = torch.randn(32, 1, 4, 4).cuda()
for e in range(epochs):
    random_input = torch.randn(32, 1, 4, 4).cuda()
    Y_pred = encoder(random_input)
    loss = critertion( Y_pred, Y )
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if epoch % 2 == 0:
        print('[Epoch:{} -- Loss:{:.4f}]'.format(epoch, loss.item()))

</code></pre>
<pre><code class="lang-auto">[Epoch:0 -- Loss: 0.8634] # loss getting updated
[Epoch:2 -- Loss: 0.8574]
</code></pre>
<pre><code class="lang-auto">for params in encoder.parameters(): #looping through the encoder to see if my weights are still the same
    print('params:' params)

params: [ [[[ 0.1433]]], [[[0.9233]]], [[[-0.1333]]], [[[-0.7586]] ] ... # weight value updates?
</code></pre>
<p>As you can see if I print out the new param after 2 epochs my weights still got updated. I would like to cancel those update and only use them as input conversion layers like toRGB or fromRGB layers in the progressive gan paper.</p>

albanD: <p>Hi,</p>
<p>This looks like l2 regularization or similar behaviour of the optimizer: all your weights are slightly closer to 0.</p>

inkplay: <p>After reading your comment I used SGD instead of Adam and set weight decay to 0 which suppose to stop L2 regularization and my weight still updates? Am I misunderstanding the concept of freezing weights? Are the weights suppose to be updated but never used or their values are not suppose to change?</p>

ptrblck: <p>We’ve had this discussion a while ago and for investigation of this effect I created a <a href="https://gist.github.com/ptrblck/d9abccd4f52b1aa6d242da3338533169">gist</a> showing how <code>optim.Adam</code> updates the parameters even without a gradient once the running estimates were set.<br>
Could you compare your code to the example and make sure no momentum etc. is set?</p>

inkplay: <p>Hi ptrblck I thought ill make it easier by switching to SGD instead and I have set everything to default values and I made sure momentum is zero, I even restarted my computer just to make sure I have a clean slate.  I still have this problem, I don’t think its just Adam because of SGD still updates my weights after the require_grad = False. Actually while I was typing this I added a print statement right after require_grad = False and it prints require_grad to True?? Did I write my code wrong?</p>
<p>My code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

encoder = nn.Sequential( nn.Conv2d(1, 4, 1), nn.Sigmoid())

for params in encoder.parameters():
    
    params.require_grad = False
    print(params.requires_grad)

</code></pre>
<p>The print statement comes after I change require_grad to False but if I print out the setting I would get 2 True outputs.</p>

SimonW: <p>require<strong>s</strong>_grad</p>
<p>you are missing an “s”</p>

inkplay: <p>Oh my god, thank you no wonder this doesn’t work I had a typo all along. So what is require_grad then? It doesn’t throw an error. Thank you so much SimonW you must be a very handsome person in real life.</p>

SimonW: <p>It’s just assigning a new attribute. For python objects, if you do <code>a.b = c</code>, <code>a.b</code> doesn’t have to exist before this.</p>

inkplay: <p>Thank you I am self taught and this was a good mistake to learn from.</p>

Julien_Jorda: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> ,<br>
I was wondering if you finally found a workaround to this problem. I am doing some Transfer Learning where I initialize a model with the parameters of another previously trained model and freeze the few last fully connected layers. Here is how I typically do it:</p>
<pre><code class="lang-auto">for name, param in self.model.named_parameters():
	#tells whether we want to use gradients for a given parameter 
	if freeze:
	    param.requires_grad = False
	    print("Freezing parameter "+name)
</code></pre>
<p>Transfer looks fine and parameters are initially identical, but when I compare the respective min,max,mean and std values for each parameter of each layer in both models,  some of the frozen instances start to vary after a few epochs. See below a case where I froze group1-4:<br>
The model I am transferring the weights from:</p>
<pre><code class="lang-auto">Name                            Min          Max          Mean           Std  
----------------------  -----------  -----------  ------------  ------------  
module.group1.0.weight  -0.13601      0.135853     0.000328239    0.078537    
module.group1.0.bias    -0.129506     0.13031     -0.000709442    0.0761818   
module.group2.0.weight  -0.0156249    0.015625    -6.8284e-06     0.00902701  
module.group2.0.bias    -0.0150966    0.0152359    0.000233364    0.00887235  
module.group3.0.weight  -0.0110485    0.0110485    8.25103e-06    0.00637962  
module.group3.0.bias    -0.0109931    0.0109642   -0.000212902    0.00620885  
module.group4.0.weight  -0.0078125    0.0078125   -1.07069e-06    0.00451099  
module.group4.0.bias    -0.0077329    0.00775102  -0.000157763    0.00451984  
module.fc1.0.weight     -0.00195312   0.00195312  -1.05901e-08    0.00112767  
module.fc1.0.bias       -0.00195279   0.0019521    5.93193e-05    0.00113513  
module.fc2.0.weight     -0.0312486    0.0312499   -2.94543e-05    0.0180225   
module.fc2.0.bias       -0.0312394    0.0289709   -0.00238465     0.0186226   
module.fc3.0.weight     -0.100976     0.0989116   -0.00164936     0.0606025   
module.fc3.0.bias       -0.059265    -0.059265    -0.059265     nan           

</code></pre>
<p>The model that I initialized thtough TL:</p>
<pre><code class="lang-auto">Name                            Min          Max          Mean           Std  
----------------------  -----------  -----------  ------------  ------------  
module.group1.0.weight  -0.136078     0.136051     0.00138295     0.0788667   
module.group1.0.bias    -0.135537     0.135878     0.00912299     0.0691942   
module.group2.0.weight  -0.0156247    0.0156249   -2.81046e-05    0.00902321  
module.group2.0.bias    -0.0151269    0.0152803    0.000945539    0.0088397   
module.group3.0.weight  -0.0110485    0.0110485   -7.81598e-06    0.00637801  
module.group3.0.bias    -0.0110323    0.0109976   -0.000282283    0.00675859  
module.group4.0.weight  -0.0078125    0.0078125   -8.4189e-07     0.00451147  
module.group4.0.bias    -0.00777942   0.00779467  -2.26952e-05    0.00466924  
module.fc1.0.weight     -0.00195312   0.00195312   1.48078e-07    0.00112768  
module.fc1.0.bias       -0.00194499   0.00195289   5.32243e-05    0.00112104  
module.fc2.0.weight     -0.0312488    0.0312494   -5.54657e-06    0.0180232   
module.fc2.0.bias       -0.0304042    0.0306912    0.00134896     0.018436    
module.fc3.0.weight     -0.0996469    0.101409    -0.00436459     0.0568807   
module.fc3.0.bias       -0.0561954   -0.0561954   -0.0561954    nan     
</code></pre>
<p>Any insight would be appreciated!</p>
<p>Thanks,</p>

ptrblck: <p>Are you freezing the parameters from the beginning and are you using e.g. weight decay?<br>
If so, could only pass the parameters which require grads to the optimizer and run the code again?</p>

Julien_Jorda: <p>I am actually freezing them from the beginning and I do use weight decay.<br>
I believe I am already passing only the parameters that require grads to the optimizer. See below:</p>
<pre><code class="lang-auto">self.optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, self.model.parameters()), lr=self.learning_rate, weight_decay=self.penalty)
</code></pre>

ptrblck: <p>Your approach looks alright.<br>
Could you post a minimal code snippet to reproduce the issue?</p>

Nicole_Yan: <p>Hi Julien,</p>
<p>Any luck or insights on this? I have similar issues: use transfer learning, freeze some layers, and the weights of those frozen layers still get updated.</p>

ptrblck: <p>Were these parameters trained before and are you using an optimizer with internal states, e.g. <code>Adam</code>?<br>
If so, note that the running internal states might still update the frozen parameters, as seen in this code snippet:</p>
<pre><code class="lang-python"># Setup
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.enc = nn.Linear(64, 10)
        self.dec = nn.Linear(10, 64)
        
    def forward(self, x):
        x = F.relu(self.enc(x))
        x = self.dec(x)

        return x


x = torch.randn(1, 64)
y = x.clone()
model = MyModel()
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=1.)

# dummy updates
for idx in range(10):
    optimizer.zero_grad()
    output = model(x)
    loss = criterion(output, y)
    loss.backward()
    optimizer.step()
    print('Iter{}, loss {}'.format(idx, loss.item()))

optimizer.zero_grad()
# Freeze encoder
for param in model.enc.parameters():
    param.requires_grad_(False)

# Store reference parameter
enc_weight0 = model.enc.weight.clone()

# Update for more iterations
for idx in range(10):
    optimizer.zero_grad()
    output = model(x)
    loss = criterion(output, y)
    loss.backward()
    optimizer.step()
    print('Iter{}, loss {}'.format(idx, loss.item()))
    print('max abs diff in enc.weight {}'.format(
        (enc_weight0 - model.enc.weight).abs().max()))
    print('sum abs grad in enc.weight {}'.format(
        model.enc.weight.grad.abs().sum()))
</code></pre>

Nicole_Yan: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I’m using pre-trained Places365-resnet50 as a base model and added a new fc layer. Only the newly added fc layer is trained to classify sun attributes. So in one pass I can predict both places 365 categories and sun attributes.</p>
<p>Here is my model:</p>
<pre><code class="lang-auto"># the architecture to use
arch = 'resnet50'

# load the pre-trained weights
model_file = '%s_places365.pth.tar' % arch
if not os.access(model_file, os.W_OK):
    weight_url = 'http://places2.csail.mit.edu/models_places365/' + '%s_places365.pth.tar' % arch
    os.system('wget ' + weight_url)
model = models.__dict__[arch](num_classes=365)
checkpoint = torch.load(model_file, map_location=lambda storage, loc: storage)
state_dict = {str.replace(k, 'module.', ''): v for k, v in checkpoint['state_dict'].items()}
model.load_state_dict(state_dict)

class CustomizedResNet(nn.Module):

    def __init__(self):
        super(CustomizedResNet, self).__init__()

        # Resnet 50 as base model
        self.base_model = model

        def hook_feature(module, input, output):
            self.feature = output

        self.base_model._modules.get('avgpool').register_forward_hook(hook_feature)

        self.scene_attr_fc = nn.Linear(2048, 102)

        # freeze weights
        for param in self.base_model.parameters():
            param.requires_grad = False

        for param in self.scene_attr_fc.parameters():
            param.requires_grad = True

    def forward(self, x):

        places365_output = self.base_model(x)

        # compute scene attributes
        # feed the outputs from avgpool to the new fc layer
        attributes_output = self.feature.view(self.feature.size(0), -1)
        attributes_output = self.scene_attr_fc(attributes_output)

        return places365_output, attributes_output

customized_model = CustomizedResNet()
</code></pre>
<p>And I only pass the parameters of the new fc layer to the optimizer.</p>
<pre><code class="lang-auto">criterion = torch.nn.BCEWithLogitsLoss()
optimizer = optim.SGD(customized_model.scene_attr_fc.parameters(), lr=learning_rate)
</code></pre>
<p>My training parts look like this:</p>
<pre><code class="lang-auto">torch.save(customized_model.state_dict(), 'before_training.pth')

for epoch in range(num_epochs):

       customized_model.train()
       for i, (inputs, labels) in enumerate(dataloader):
            inputs = inputs.to(device)
            labels = labels.to(device)

            # zero the parameter gradients
            optimizer.zero_grad()

            with torch.set_grad_enabled(True):
                _, scene_attr_outputs = customized_model(inputs)
                loss = criterion(scene_attr_outputs, labels)
                loss.backward()
                optimizer.step()
       torch.save(customized_model.state_dict(), 'model_saved_at_epoch_%s.pth' % epoch)
</code></pre>
<p>And my testing parts look like this:</p>
<pre><code class="lang-auto">data_transforms = {
    'test': transforms.Compose([
        transforms.Resize((224,224)),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
}

customized_model.load_state_dict(torch.load('model_saved_at_epoch_9.pth'))
customized_model.eval()

images_test = ['stone.jpg']

for img_path in images_test:
    # load test image
    img = Image.open(img_path).convert('RGB')
    img = data_transforms['test'](img)
    img = img.to(device)

    # prediction
    places365_outputs, scene_attr_outputs = customized_model.forward(img.unsqueeze(0))

    # prediction for places365
    # print(places365_outputs.shape) -&gt; torch.Size([1, 365])
    h_x = F.softmax(places365_outputs, 1).data.squeeze()
    probs, idx = h_x.sort(0, True)
    print('places 365 prediction on {}'.format(img_path))
    for i in range(0, 5):
       # classes stores all 365 labels
       print('{:.3f} -&gt; {}'.format(probs[i], classes[idx[i]]))
</code></pre>
<p>The problem is, if I use the models saved at different epoch to predict the same image, the prediction of places 365 is changing even if I already freeze all weights for places 365 branch.<br>
For example,<br>
If I use the model saved before any training happens, its prediction is</p>
<pre><code class="lang-auto">places 365 prediction on stone.jpg
0.298 -&gt; coast
0.291 -&gt; ocean
0.172 -&gt; beach
0.067 -&gt; ice_floe
0.051 -&gt; sky
</code></pre>
<p>If I use model_saved_at_epoch_3.pth, it gives</p>
<pre><code class="lang-auto">places 365 prediction on stone.jpg
0.308 -&gt; coast
0.259 -&gt; ocean
0.132 -&gt; beach
0.107 -&gt; sky
0.041 -&gt; cliff
</code></pre>
<p>model_saved_at_epoch_13.pth gives:</p>
<pre><code class="lang-auto">places 365 prediction on stone.jpg
0.295 -&gt; coast
0.234 -&gt; ocean
0.152 -&gt; sky
0.111 -&gt; beach
0.047 -&gt; cliff
</code></pre>
<p>I even compared the weights of the base_model after every epoch to the original weights, and it looks like the weights didn’t change:</p>
<pre><code class="lang-auto"># before training occurs
original_weights = []
for name, param in customized_model.base_model.named_parameters():
    original_weights.append(param.clone())

for epoch in range(num_epochs):
       # training .....
       max_abs_diff_sum = 0
       idx = 0
       for epoch_name, epoch_param in customized_model.base_model.named_parameters():
            max_abs_diff_sum += (original_weights[idx] - epoch_param).abs().max()
            idx += 1
       print(max_abs_diff_sum)  # all print tensor(0., device='cuda:0'). So I think the weights of base_model didnt change
</code></pre>
<p>Do you have any idea on why the probability distribution is changing even if I froze all weights for the places 365 branch (and it also looks like the base_model weights are the same)?<br>
Thank you.</p>

ptrblck: <p>If you are using batch norm layers in the base model, the running estimates will still be updated even if you’ve frozen the affine parameters.<br>
To fix the running stats, you would have to call <code>.eval()</code> on all batch norm layers.<br>
Also, dropout layers might still be active, which could explain the different results.<br>
You could also call <code>.eval()</code> on all dropout layers or alternatively on the <code>self.base_model</code> to disable these effects.</p>

Nicole_Yan: <p>Fixed! Thank you so much!</p>

hoangcuong2011: <p>Hi, I found this both interesting and crucial. I want to share my experience here as well and hope it is useful.</p>
<p>Let us assume we have a variable B. I have seen case where I explicitly set B.require_grad = False but B is still updated during fine-tuning. This could happen even when I put operations involved with B within torch.no_grad().</p>
<p>My solution is to modify the code declaration of B in the code. Something like this:</p>
<blockquote>
<p>self.register_buffer(“B”, torch.ones(the shape of B looks like), requires_grad=False))</p>
</blockquote>
<p>With that when I load a pretrained model I noticed B is loaded properly. More importantly, when I fine-tune the model I noticed B is indeed frozen. This is a bit manual but it works.</p>