Not getting grad on a new parameter

username: thyeros
cooked: <p>I have a training flow where I need to add a new parameter to a module during training like below.</p>
<pre><code class="lang-auto">for epoch in range(10):
  if epoch==5:
    module.register_parameter('t',torch.nn.Parameter(torch.Tensor([0.05])))
    opt.param_groups.append(copy.deepcopy(opt.param_groups[-1]))
    opt.param_groups[-1]['params'] = [module.t]

  loss= module.forward() #use t param from epoch=5
  loss.backward()
....

</code></pre>
<p>But, I don’t see the parameter ‘t’ being updated, and find that module.t.grad=None. Is there any other step  s I need take to add a parameter on the fly?</p>

username: ptrblck
cooked: <p>Could you post a minimal and executable code snippet to reproduce the issue, please?</p>

username: thyeros
cooked: <p>thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> I cooked up a simple example. It seems grad is there, but the model.l1.t is not being updated??</p>
<aside class="quote no-group quote-modified" data-username="thyeros" data-post="1" data-topic="144448">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thyeros/48/12827_2.png" class="avatar"> thyeros:</div>
<blockquote>
<pre><code class="lang-auto">from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
from torch.optim.lr_scheduler import StepLR
import copy

class Net(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.l1 = nn.Linear(28 * 28, num_classes)
        self.relu1 = nn.ReLU(inplace=True)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = self.l1(x)
        
        if hasattr(self.l1, 't'): 
            x = x*self.l1.t
        
        return self.relu1(x)

def train(model, device, train_loader, opt, epoch):
    model.train()
    for batch_idx, (data, target) in enumerate(train_loader):
        data, target = data.to(device), target.to(device)
          
        if hasattr(model.l1, 't'): 
            print(model.l1.t, '*grad*',model.l1.t.grad)
                    
        opt.zero_grad()
        output = model(data)
        loss = F.nll_loss(output, target)
        loss.backward()
        opt.step()
 
        if batch_idx % 10 == 9:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                100. * batch_idx / len(train_loader), loss.item()))
            
            break
 

def main():
                                  
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    train_kwargs = {'batch_size': 64, 'shuffle': True} 

    transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
    dataset1 = datasets.MNIST('../data', train=True, download=True,  transform=transform)
 
    train_loader = torch.utils.data.DataLoader(dataset1,**train_kwargs)

    model = Net().to(device)
    opt = optim.Adadelta(model.parameters(), lr=0.001)

    scheduler = StepLR(opt, step_size=1, gamma=0.7)
    for epoch in range(0, 4):
        
        if epoch==2:
            model.l1.register_parameter('t',torch.nn.Parameter(torch.Tensor([0.05]).to(device)))
            opt.param_groups.append(copy.deepcopy(opt.param_groups[-1]))
            opt.param_groups[-1]['params'] = [model.l1.t]
            
            print(opt)
        
        train(model, device, train_loader, opt, epoch)
        scheduler.step()


if __name__ == '__main__':
    main()
</code></pre>
</blockquote>
</aside>

username: thyeros
cooked: <p>is this problem specific to Adadelta? with a simple SGD, it seems working…</p>

username: ptrblck
cooked: <p>I’m not sure if your manual <code>param_groups</code> manipulation works and would recommend to use <code>add_param_group</code> instead:</p>
<pre><code class="lang-python">model.l1.register_parameter('t',torch.nn.Parameter(torch.Tensor([0.05]).to(device)))
opt.add_param_group({'params': [model.l1.t]})
</code></pre>
<p>Afterwards I get this output:</p>
<pre><code class="lang-python">Train Epoch: 0 [576/60000 (1%)]	Loss: -0.093652
Train Epoch: 1 [576/60000 (1%)]	Loss: -0.130155
Adadelta (
Parameter Group 0
    eps: 1e-06
    initial_lr: 0.001
    lr: 0.00049
    rho: 0.9
    weight_decay: 0

Parameter Group 1
    eps: 1e-06
    lr: 0.001
    rho: 0.9
    weight_decay: 0
)
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* None
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1296], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1662], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.0712], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1003], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.0827], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.0788], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1489], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1556], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1602], device='cuda:0')
Train Epoch: 2 [576/60000 (1%)]	Loss: -0.007945
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1588], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.0828], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1081], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1140], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1470], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1953], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1221], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.1119], device='cuda:0')
Parameter containing:
tensor([0.0500], device='cuda:0', requires_grad=True) *grad* tensor([-0.0748], device='cuda:0')
Parameter containing:
tensor([0.0501], device='cuda:0', requires_grad=True) *grad* tensor([-0.1101], device='cuda:0')
Train Epoch: 3 [576/60000 (1%)]	Loss: -0.008487
</code></pre>
<p>which shows that the new parameter is updated.</p>

username: thyeros
cooked: <p>Apparently, both add_param_group and the way I add new params work fine. The diff is that using add_param_group will use the initial state, rather than the last state.</p>
<p>Adding new params with advanced optimizers seems not working properly, perhaps these optimizers are updated only the registered parameters with proper curvature information? (newly added ones don’t have any accumulated previous values).</p>