ERROR:optimizer got an empty parameter list

alan_ayu: <p>I am trying to build network with a new structure,The model is built with two separated class.When I run it,the error says that :optimizer got an empty parameter list.I am new to PyTorch and I don’t know what causes the error.Can you give me some suggestions?Thank you!</p>

fmassa: <p>This happens because <code>model.parameters()</code> is empty.<br>
It might probably happen because all your parameters are inside a list which is attributed to the model, and pytorch can’t find them. Something like</p>
<pre><code class="lang-python">self.myparameters = [Parameter1, Parameter2, ...]
</code></pre>
<p>If that is the case, then you should use <code>nn.ParameterList</code> instead.</p>
<pre><code class="lang-python">self.myparameters = nn.ParameterList(Parameter1, Parameter2, ...)
</code></pre>

alan_ayu: <p>Yes it works now!<br>
Thank you very much</p>

avisek_lahiri: <p>Hi. I am also facing a similar kind of scenario. I am trying to implement a simple GAN with generator and discriminator being respective classes.</p>
<pre><code class="lang-auto"># Generator Neural Network Model (1 hidden layer)
class GNet(nn.Module):

    def __init__(self, input_size, hidden_size, image_size):
        super(GNet, self).__init__()

        self.model = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, image_size),
        )
    def forward(self, x):

        return self.model(x)

# Discriminator Neural Network Model (2 hidden layer)

class DNet(nn.Module):
    def __init__(self, image_size, hidden_size):

        super(DNet, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(image_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, num_classes),
        )

    def forward(self, x):
      return self.model(x)


G = GNet(input_size, hidden_size, image_size)
D = DNet(image_size, hidden_size)
G_params = G.parameters()
D_params = D.parameters()
G_solver = optim.Adam(G_params, lr=1e-3)
D_solver = optim.Adam(D_params, lr=1e-3)
</code></pre>
<p>I am getting the error message in the line of G_solver:</p>
<pre><code class="lang-auto">ValueError: optimizer got an empty parameter list
</code></pre>
<p>Why I am I not getting the parameter list even if I am calling D.parameters()</p>

smth: <p>Do:</p>
<pre><code class="lang-auto">G_params = list(G.parameters())
D_params = list(D.parameters())
</code></pre>
<p><code>.parameters()</code> is a generator, and probably for debugging purposes you are pre-populating it somewhere.</p>

Insane: <pre><code class="lang-auto">class Netz ( nn.Module ):
    def _init_(self):
        super ( Netz, self ).__init__ ()
        self.conv1 = nn.Conv2d ( 1, 10, kernel_size=5 )
        self.conv2 = nn.Conv2d ( 10, 20, kernel_size=5 )
        self.conv_dropout = nn.Dropout2d ()
        self.fc1 = nn.Linear ( 320, 60 )
        self.fc2 = nn.Linear ( 60, 10 )

    def forward(self, x):
        x = self.conv1 ( x )
        x = F.max_pool2D ( x, 2 )
        x = F.relu ( x )
        x = self.conv2 ( x )
        x = self.conv_dropout ( x )
        x = F.max_pool2D ( x, 2 )
        x = F.relu ( x )
        x = x.view ( -1, 320 )
        x = F.relu ( self.fc1 ( x ) )
        x = self.fc2 ( x )
        return F.log_softmax ( x )

model = Netz()
model.cuda()

optimizer = optim.SGD( model.parameters (), lr=0.1, momentum=0.8 )
def train(epoch):
    model.train ()
    for batch_id, (data, target) in enumerate ( train_data ):
        data = data.cuda ()
        target = target.cuda ()
        data = Variable ( data )
        target: Variable = Variable ( target )
        optimizer.zero_grad ()
        out = model ( data )
        criterion = F.nll_loss
        loss = criterion ( out, target )
        loss.backward ()
        optimizer.step ()
        print ( 'Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format ( epoch, batch_id * len ( data ),
                                                                           len ( train_data.dataset ),
                                                                           100. * batch_id / len ( train_data ),
                                                                           loss.data[0] ) )
    for epoch in range ( 1, 30 ):
        train ( epoch )
</code></pre>
<p>the same Error here:<br>
ValueError: optimizer got an empty parameter list --&gt; optimizer = optim.SGD( model.parameters (), lr=0.1, momentum=0.8 )<br>
I have change to: optimizer = optim.SGD(list( model.parameters ()), lr=0.1, momentum=0.8 ) -&gt; also the same Error<br>
Where is the failure?</p>

Insane: <p>OK, done! i got the failure!<br>
My def <em>init</em>(self): was incorrect typed</p>

Novak: <p>I am having the same issue with a different architecture.  Specifically, I am trying to build a stacked LSTM, where each layer may have a different number of units.  It would be very convenient to pass a list of layer sizes as an argument and have the same function build the network as appropriate.  Every attempt has failed:</p>
<ul>
<li>
<p>The most obvious way is to build lists of ‘torch.nn.LSTM’ and hidden layers, and manage them directly.  This fails, I think, because they are “hidden” in a list, even though defined in the <strong>init</strong> routine.</p>
</li>
<li>
<p>Similarly, but for completeness, building a dict of layers with integer keys failed, for what I believe is the same reason.</p>
</li>
<li>
<p>A rather horrible attempt to use ‘vars(self)’ to force-construct meaningful variable names failed in a more complicated way, and honestly, it’s a terrible idea anyway.</p>
</li>
</ul>
<p>What is the accepted way to do what I’m trying to do, without re-implementing the whole LSTM architecture myself?  Is there a way to force the registration of the LSTM objects with the parameters list?</p>

Shikai_Qiu: <p>I made the same error with you, so embarrassed…</p>

111137: <p>Hi,</p>
<p>I am newer, I tried to make myown operation with PyTorch, then I meet this error.<br>
So, I try to code as follows;</p>
<pre><code class="lang-python">self.myparameters = nn.ParameterList(self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2)
</code></pre>
<p>This is in <strong>init(self)</strong> and all parameters are like this;</p>
<pre><code class="lang-python">self.w     = torch.randn((NUM_INPUT, NUM_HIDDEN), requires_grad=True)
</code></pre>
<p>Error Message is as follows;</p>
<pre><code class="lang-python">TypeError                                 Traceback (most recent call last)
&lt;ipython-input-75-31d214f55eb8&gt; in &lt;module&gt;()
     33       return out2
     34 
---&gt; 35 model = Model().to(device)
     36 optimizer = optim.SGD(model.parameters(), lr=lr)
     37 criterion = nn.CrossEntropyLoss()

&lt;ipython-input-75-31d214f55eb8&gt; in __init__(self)
     24         self.out_b2= torch .randn((NUM_CLASSES), requires_grad=True)
     25 
---&gt; 26         self.myparameters = nn.ParameterList(self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2)
     27 
     28     def forward(self, x):

TypeError: __init__() takes from 1 to 2 positional arguments but 6 were given
</code></pre>
<p>What does the message mean? and how to solve?</p>

liyz15: <p>Not sure what you are trying to do, but the error is because <code>nn.ParameterList</code> takes an iterable as input, see <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.ParameterList" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#torch.nn.ParameterList</a>, try<br>
<code>self.myparameters = nn.ParameterList([self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2])</code></p>

111137: <p>Sure, I did.<br>
My model is;</p>
<pre><code class="lang-python">class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        
        # Gate-Weight
        self.w     = torch.randn((NUM_INPUT,  NUM_HIDDEN), requires_grad=True)
    
        # Gate-Selector
        self.sel   = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Input Vector
        self.fw_x  = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Output Vector
        self.fw_h  = torch.zeros(NUM_INPUT, NUM_HIDDEN)
    
        # Output Layer
        self.out_w1= torch.randn((NUM_HIDDEN),             requires_grad=True)
        self.out_b1= torch.randn((NUM_INPUT),              requires_grad=True)
        self.out_w2= torch.randn((NUM_CLASSES, NUM_INPUT), requires_grad=True)
        self.out_b2= torch.randn((NUM_CLASSES),            requires_grad=True)
        
        #self.myparameters = nn.ParameterList([self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2])

    def forward(self, x):
      fw_prop(self, x)
      out1 = torch.matmul(self.fw_h, self.out_w1) + self.out_b1
      out2 = torch.matmul(self.out_w2, out1) + self.out_b2

      return out2
    
model = Model().to(device)
optimizer = optim.SGD(model.parameters(), lr=lr)
criterion = nn.CrossEntropyLoss()
</code></pre>
<p>Then;</p>
<pre><code class="lang-python">---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-12-c94710f76e27&gt; in &lt;module&gt;()
     36 model = Model().to(device)
     37 #model.myparameters = torch.nn.Parameter(model.w, model.out_w1, model.out_b1, model.out_w2, model.out_b2)
---&gt; 38 optimizer = optim.SGD(model.parameters(), lr=lr)
     39 criterion = nn.CrossEntropyLoss()

1 frames
/usr/local/lib/python3.6/dist-packages/torch/optim/optimizer.py in __init__(self, params, defaults)
     43         param_groups = list(params)
     44         if len(param_groups) == 0:
---&gt; 45             raise ValueError("optimizer got an empty parameter list")
     46         if not isinstance(param_groups[0], dict):
     47             param_groups = [{'params': param_groups}]

ValueError: optimizer got an empty parameter list
</code></pre>
<p>So I did both of</p>
<pre><code class="lang-python">ParameterList(...)
</code></pre>
<p>and</p>
<pre><code class="lang-python">ParameterList([...])
</code></pre>
<p>Last one has error of;</p>
<pre><code class="lang-python">---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
&lt;ipython-input-13-87872cf3c699&gt; in &lt;module&gt;()
     34       return out2
     35 
---&gt; 36 model = Model().to(device)
     37 optimizer = optim.SGD(model.parameters(), lr=lr)
     38 criterion = nn.CrossEntropyLoss()

4 frames
&lt;ipython-input-13-87872cf3c699&gt; in __init__(self)
     25         self.out_b2= torch.randn((NUM_CLASSES),            requires_grad=True)
     26 
---&gt; 27         self.myparameters = nn.ParameterList([self.w, self.out_w1, self.out_b1, self.out_w2, self.out_b2])
     28 
     29     def forward(self, x):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in __init__(self, parameters)
    360         super(ParameterList, self).__init__()
    361         if parameters is not None:
--&gt; 362             self += parameters
    363 
    364     def _get_abs_string_index(self, idx):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in __iadd__(self, parameters)
    389 
    390     def __iadd__(self, parameters):
--&gt; 391         return self.extend(parameters)
    392 
    393     def __dir__(self):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py in extend(self, parameters)
    416         offset = len(self)
    417         for i, param in enumerate(parameters):
--&gt; 418             self.register_parameter(str(offset + i), param)
    419         return self
    420 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in register_parameter(self, name, param)
    155             raise TypeError("cannot assign '{}' object to parameter '{}' "
    156                             "(torch.nn.Parameter or None required)"
--&gt; 157                             .format(torch.typename(param), name))
    158         elif param.grad_fn:
    159             raise ValueError(

TypeError: cannot assign 'torch.FloatTensor' object to parameter '0' (torch.nn.Parameter or None required)
</code></pre>
<p>The iterative means of;</p>
<pre><code class="lang-python">    417         for i, param in enumerate(parameters):
--&gt; 418             self.register_parameter(str(offset + i), param)
</code></pre>
<p>Is my understanding a correct?</p>

111137: <pre><code class="lang-python">        self.fc1 = nn.Linear(NUM_HIDDEN, 1)
        self.fc2 = nn.Linear(NUM_INPUT, NUM_CLASSES)
</code></pre>
<p>with transpose, then no error, so probably error is on a shape.</p>

David_Whyte: <p>You can do this instead <code>optimizer = optim.SGD( params = model.parameters (), lr=0.1, momentum=0.8 )</code></p>

David_Whyte: <p>You should do this <code>G_solver = optim.Adam(params = G_params, lr=1e-3)</code></p>

Erun: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/ec8c36a0271e6f1ae316136fa8743be7c3b7059a.png" data-download-href="https://discuss.pytorch.org/uploads/default/ec8c36a0271e6f1ae316136fa8743be7c3b7059a" title="%E5%9B%BE%E7%89%87"><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/ec8c36a0271e6f1ae316136fa8743be7c3b7059a.png" alt="%E5%9B%BE%E7%89%87" data-base62-sha1="xKB5vnRLPMc5AbM1EoyQOueB9ZU" width="595" height="500" data-dominant-color="312E2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">%E5%9B%BE%E7%89%87</span><span class="informations">807×678 36.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<aside class="quote no-group" data-username="Insane" data-post="8" data-topic="1501" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/i/47e85d/48.png" class="avatar"> Insane:</div>
<blockquote>
<p>OK, done! i got the failure!<br>
My def <em>init</em>(self): was incorrect typed</p>
</blockquote>
</aside>
<p>How can i solve it ? thank you</p>

ptrblck: <p>Fix the typo in <code>def __init__</code> (missing i) and it should work.</p>

moaz_taha: <p>Same issue, was using <code>__init_</code>  instead of <code>__init__</code><br>
very hard to see lol</p>

mikey_t: <p>I got this error because I failed to properly use <code>self.</code> in the <code>__init__</code> of my <code>nn.Module</code> class.</p>
<p>For example, I used <code>layer1 = nn.Linear(input_size,hidden_size)</code></p>
<p>instead of <code>self.layer1 = nn.Linear(input_size,hidden_size)</code>.</p>
<p>Easy mistake to make, but it took me a while to figure out</p>

Rafay: <p><strong>Hello I am also facing the same issue. Can anyone please help me.</strong></p>
<p>from turtle import forward<br>
import torch<br>
import torch.nn as nn</p>
<p>class NeuralNet(nn.Module):<br>
def <strong>init</strong>(self, input_size, hidden_size, num_classes):<br>
super(NeuralNet, self).<strong>init</strong>()<br>
self.l1 = nn.AdaptiveMaxPool2d(input_size, hidden_size)<br>
self.l2 = nn.AdaptiveMaxPool2d(hidden_size, hidden_size)<br>
self.l1 = nn.AdaptiveMaxPool2d(hidden_size, num_classes)</p>
<pre><code>    self.relu = nn.ReLU()
    
def forward(self, x):
    out = self.l1(x)
    out = self.relu(out)
    out = self.l2(out)
    out = self.relu(out)
    out = self.l3(out)
    #no activation and no softmax
    return out
</code></pre>
<p>[nltk_data] Downloading package punkt to<br>
[nltk_data]     C:\Users\Hp\AppData\Roaming\nltk_data…<br>
[nltk_data]   Package punkt is already up-to-date!<br>
Traceback (most recent call last):<br>
File “c:/Users/Hp/Desktop/rafay pytorch/train.py”, line 77, in <br>
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)<br>
File “C:\Users\Hp\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\optim\adam.py”, line 90, in <strong>init</strong><br>
super(Adam, self).<strong>init</strong>(params, defaults)<br>
File “C:\Users\Hp\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\optim\optimizer.py”, line 49, in<br>
<strong>init</strong><br>
raise ValueError(“optimizer got an empty parameter list”)<br>
ValueError: optimizer got an empty parameter list</p>