ERROR:optimizer got an empty parameter list

alan_ayu: I am trying to build network with a new structure,The model is built with two separated class.When I run it,the error says that :optimizer got an empty parameter list.I am new to PyTorch and I don’t know what causes the error.Can you give me some suggestions?Thank you!

fmassa: <SCODE>self.myparameters = [Parameter1, Parameter2, ...]
<ECODE> <SCODE>self.myparameters = nn.ParameterList(Parameter1, Parameter2, ...)
<ECODE>

alan_ayu: 

avisek_lahiri: Hi. I am also facing a similar kind of scenario. I am trying to implement a simple GAN with generator and discriminator being respective classes. <SCODE># Generator Neural Network Model (1 hidden layer)
class GNet(nn.Module):

    def __init__(self, input_size, hidden_size, image_size):
        super(GNet, self).__init__()

        self.model = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, image_size),
        )
    def forward(self, x):

        return self.model(x)

# Discriminator Neural Network Model (2 hidden layer)

class DNet(nn.Module):
    def __init__(self, image_size, hidden_size):

        super(DNet, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(image_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, num_classes),
        )

    def forward(self, x):
      return self.model(x)


G = GNet(input_size, hidden_size, image_size)
D = DNet(image_size, hidden_size)
G_params = G.parameters()
D_params = D.parameters()
G_solver = optim.Adam(G_params, lr=1e-3)
D_solver = optim.Adam(D_params, lr=1e-3)
<ECODE> I am getting the error message in the line of G_solver: <SCODE>ValueError: optimizer got an empty parameter list
<ECODE> Why I am I not getting the parameter list even if I am calling D.parameters()

smth: Do: <SCODE>G_params = list(G.parameters())
D_params = list(D.parameters())
<ECODE>

Insane: <SCODE>class Netz ( nn.Module ):
    def _init_(self):
        super ( Netz, self ).__init__ ()
        self.conv1 = nn.Conv2d ( 1, 10, kernel_size=5 )
        self.conv2 = nn.Conv2d ( 10, 20, kernel_size=5 )
        self.conv_dropout = nn.Dropout2d ()
        self.fc1 = nn.Linear ( 320, 60 )
        self.fc2 = nn.Linear ( 60, 10 )

    def forward(self, x):
        x = self.conv1 ( x )
        x = F.max_pool2D ( x, 2 )
        x = F.relu ( x )
        x = self.conv2 ( x )
        x = self.conv_dropout ( x )
        x = F.max_pool2D ( x, 2 )
        x = F.relu ( x )
        x = x.view ( -1, 320 )
        x = F.relu ( self.fc1 ( x ) )
        x = self.fc2 ( x )
        return F.log_softmax ( x )

model = Netz()
model.cuda()

optimizer = optim.SGD( model.parameters (), lr=0.1, momentum=0.8 )
def train(epoch):
    model.train ()
    for batch_id, (data, target) in enumerate ( train_data ):
        data = data.cuda ()
        target = target.cuda ()
        data = Variable ( data )
        target: Variable = Variable ( target )
        optimizer.zero_grad ()
        out = model ( data )
        criterion = F.nll_loss
        loss = criterion ( out, target )
        loss.backward ()
        optimizer.step ()
        print ( 'Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format ( epoch, batch_id * len ( data ),
                                                                           len ( train_data.dataset ),
                                                                           100. * batch_id / len ( train_data ),
                                                                           loss.data[0] ) )
    for epoch in range ( 1, 30 ):
        train ( epoch )
<ECODE>

Insane: 

Novak: I am having the same issue with a different architecture.  Specifically, I am trying to build a stacked LSTM, where each layer may have a different number of units.  It would be very convenient to pass a list of layer sizes as an argument and have the same function build the network as appropriate.  Every attempt has failed: 
The most obvious way is to build lists of ‘torch.nn.LSTM’ and hidden layers, and manage them directly.  This fails, I think, because they are “hidden” in a list, even though defined in the init routine.
 
Similarly, but for completeness, building a dict of layers with integer keys failed, for what I believe is the same reason.
 
A rather horrible attempt to use ‘vars(self)’ to force-construct meaningful variable names failed in a more complicated way, and honestly, it’s a terrible idea anyway.
 What is the accepted way to do what I’m trying to do, without re-implementing the whole LSTM architecture myself?  Is there a way to force the registration of the LSTM objects with the parameters list?

Shikai_Qiu: I made the same error with you, so embarrassed…

111137: Hi, <SCODE>self.myparameters = nn.ParameterList(self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2)
<ECODE> <SCODE>self.w     = torch.randn((NUM_INPUT, NUM_HIDDEN), requires_grad=True)
<ECODE> Error Message is as follows; <SCODE>TypeError                                 Traceback (most recent call last)
<ipython-input-75-31d214f55eb8> in <module>()
     33       return out2
     34 
---> 35 model = Model().to(device)
     36 optimizer = optim.SGD(model.parameters(), lr=lr)
     37 criterion = nn.CrossEntropyLoss()

<ipython-input-75-31d214f55eb8> in __init__(self)
     24         self.out_b2= torch .randn((NUM_CLASSES), requires_grad=True)
     25 
---> 26         self.myparameters = nn.ParameterList(self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2)
     27 
     28     def forward(self, x):

TypeError: __init__() takes from 1 to 2 positional arguments but 6 were given
<ECODE> What does the message mean? and how to solve?

liyz15: 

111137: <SCODE>class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        
        # Gate-Weight
        self.w     = torch.randn((NUM_INPUT,  NUM_HIDDEN), requires_grad=True)
    
        # Gate-Selector
        self.sel   = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Input Vector
        self.fw_x  = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Output Vector
        self.fw_h  = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Output Layer
        self.out_w1= torch.randn((NUM_HIDDEN),             requires_grad=True)
        self.out_b1= torch.randn((NUM_INPUT),              requires_grad=True)
        self.out_w2= torch.randn((NUM_CLASSES, NUM_INPUT), requires_grad=True)
        self.out_b2= torch.randn((NUM_CLASSES),            requires_grad=True)
        
        #self.myparameters = nn.ParameterList([self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2])

    def forward(self, x):
      fw_prop(self, x)
      out1 = torch.matmul(self.fw_h, self.out_w1) + self.out_b1
      out2 = torch.matmul(self.out_w2, out1) + self.out_b2

      return out2
    
model = Model().to(device)
optimizer = optim.SGD(model.parameters(), lr=lr)
criterion = nn.CrossEntropyLoss()
<ECODE> Then; <SCODE>---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-c94710f76e27> in <module>()
     36 model = Model().to(device)
     37 #model.myparameters = torch.nn.Parameter(model.w, model.out_w1, model.out_b1, model.out_w2, model.out_b2)
---> 38 optimizer = optim.SGD(model.parameters(), lr=lr)
     39 criterion = nn.CrossEntropyLoss()

1 frames
/usr/local/lib/python3.6/dist-packages/torch/optim/optimizer.py in __init__(self, params, defaults)
     43         param_groups = list(params)
     44         if len(param_groups) == 0:
---> 45             raise ValueError("optimizer got an empty parameter list")
     46         if not isinstance(param_groups[0], dict):
     47             param_groups = [{'params': param_groups}]

ValueError: optimizer got an empty parameter list
<ECODE> So I did both of <SCODE>ParameterList(...)
<ECODE> and <SCODE>ParameterList([...])
<ECODE> Last one has error of; <SCODE>---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-87872cf3c699> in <module>()
     34       return out2
     35 
---> 36 model = Model().to(device)
     37 optimizer = optim.SGD(model.parameters(), lr=lr)
     38 criterion = nn.CrossEntropyLoss()

4 frames
<ipython-input-13-87872cf3c699> in __init__(self)
     25         self.out_b2= torch.randn((NUM_CLASSES),            requires_grad=True)
     26 
---> 27         self.myparameters = nn.ParameterList([self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2])
     28 
     29     def forward(self, x):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in __init__(self, parameters)
    360         super(ParameterList, self).__init__()
    361         if parameters is not None:
--> 362             self += parameters
    363 
    364     def _get_abs_string_index(self, idx):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in __iadd__(self, parameters)
    389 
    390     def __iadd__(self, parameters):
--> 391         return self.extend(parameters)
    392 
    393     def __dir__(self):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in extend(self, parameters)
    416         offset = len(self)
    417         for i, param in enumerate(parameters):
--> 418             self.register_parameter(str(offset + i), param)
    419         return self
    420 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in register_parameter(self, name, param)
    155             raise TypeError("cannot assign '{}' object to parameter '{}' "
    156                             "(torch.nn.Parameter or None required)"
--> 157                             .format(torch.typename(param), name))
    158         elif param.grad_fn:
    159             raise ValueError(

TypeError: cannot assign 'torch.FloatTensor' object to parameter '0' (torch.nn.Parameter or None required)
<ECODE> The iterative means of; <SCODE>    417         for i, param in enumerate(parameters):
--> 418             self.register_parameter(str(offset + i), param)
<ECODE> Is my understanding a correct?

111137: <SCODE>        self.fc1 = nn.Linear(NUM_HIDDEN, 1)
        self.fc2 = nn.Linear(NUM_INPUT, NUM_CLASSES)
<ECODE> with transpose, then no error, so probably error is on a shape.

David_Whyte: 

David_Whyte: 

Erun: How can i solve it ? thank you

ptrblck: 

moaz_taha: 

mikey_t: Easy mistake to make, but it took me a while to figure out

Rafay: Hello I am also facing the same issue. Can anyone please help me. <SCODE>    self.relu = nn.ReLU()
    
def forward(self, x):
    out = self.l1(x)
    out = self.relu(out)
    out = self.l2(out)
    out = self.relu(out)
    out = self.l3(out)
    #no activation and no softmax
    return out
<ECODE>