Parameters() method returns empty list when applied on custom neural network

username: FenryrMKIII
cooked: <p>Hello,</p>
<p>I am doing a homemade project in order to better learn the details of PyTorch and be able to write my own networks, training routine, …</p>
<p>As part of this project, I am creating a replica of LeNet-5 neural network.</p>
<p>The code for custom-made layer looks like this :</p>
<pre><code class="lang-auto">from collections import Sequence
import torch.nn as nn
import torch.nn.functional as F
import numpy as np

class conv2DAveragePool(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride, padding,
                 activation=nn.Tanh()):
        super().__init__()

        def parse_2Length_Sequence(x):
            if isinstance(x, (Sequence, np.ndarray)):
                if len(x)==2:
                    item1 = x[0]
                    item2 = x[1]
                else:
                    raise ValueError('activation parameter has an unexpected'
                                    ' sequence length of ' + 
                                    len(x) + ' while one expects'
                                    'either a str or a 2-length sequence') #TODO is this correct way to catch such mistakes i.e. the if and the raise ? and the type of raise ?
                                                                            #TODO is this the correct kind of error message to send to the user ?
            else:
                item1 = item2 = x

            return item1, item2
        
        # Parameters
        self.in_channels = in_channels
        self.kernel_size = list(parse_2Length_Sequence(kernel_size)) #TODO if I have to check all arguments ... It this usually the case?? good practice ??
        self.out_channels = out_channels
        self.kernel_size = list(parse_2Length_Sequence(kernel_size))
        self.stride = list(parse_2Length_Sequence(stride))
        self.padding = list(parse_2Length_Sequence(padding))
        self.activation = list(parse_2Length_Sequence(activation))

        self.layers = nn.Sequential(
            nn.Conv2d(self.in_channels, self.out_channels, 
                      self.kernel_size[0], self.stride[0], 
                      self.padding[0], bias=True), 
            self.activation[0],
            nn.AvgPool2d(self.kernel_size[1], 
                          self.stride[1], self.padding[1]),
            self.activation[1]
                                         )
    
    def forward(self, x):
        return self.layers(x)
</code></pre>
<p>while the code for the LeNet-5 network looks like this :</p>
<pre><code class="lang-auto">class LeNet5(nn.Module):
    def __init__(self, in_channels=1):
        super().__init__()
        self.in_channels = in_channels

        self.layers = OrderedDict([('convLayers',nn.Sequential(
                                                layers.conv2DAveragePool(self.in_channels,6, 
                                                                        [5,2], [1,2], 0),
                                                layers.conv2DAveragePool(6, 16, 
                                                                        [5, 2], [1, 2], 0),
                                                nn.Conv2d(16, 120, 5,  1, 0),
                                                nn.Tanh()
                                                                )
                                    ) ,
                                    ('fcLayers',nn.Sequential(
                                                nn.Linear(120, 84),
                                                nn.Tanh(),
                                                nn.Linear(84,10),
                                                nn.Softmax(dim=0)
                                                            )
                                    )
                                    ])

    def forward(self, x):
        out = self.layers['convLayers'](x)
        out = out.view(-1, out.shape[1]*out.shape[2]*out.shape[3])
        out = self.layers['fcLayers'](out)
        return out
</code></pre>
<p>The network works as expected regarding forward function. However, I am now trying to build the training step. And in the training step, most tutorial from PyTorch I see uses the <code>.parameters()</code> method of <code>nn.module</code> class to get the optimizer parameters so for me I would for example use :</p>
<p><code>optimizer = optim.SGD(arch.parameters(), lr=0.001, momentum=0.9)</code></p>
<p>where <code>arch</code> is an instance of my homemade LeNet5 class. However, calling the <code>list(arch.parameters())</code> returns an empty list <code>[]</code>.</p>
<p>So somehow, I assume I miss a step in the definition of my network and I could not find this specifically discuss in the tutorials. Any help would be appreciated <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: albanD
cooked: <p>Hi,</p>
<p>When working with <code>nn.Module</code>, we only check the child <code>nn.Module</code>s when looking for parameters.<br>
In particular, if you store your child modules in python dictionaries or lists, they won’t be found. You can use builtin utils like <code>ModuleDict</code> or <code>ModuleList</code> (see <a href="https://pytorch.org/docs/stable/nn.html?highlight=moduledict#torch.nn.ModuleDict">doc</a>) to store collections of Modules and still have them detected.</p>

username: FenryrMKIII
cooked: <p>Hello and thank you for the reply !</p>
<p>Could you describe me what you mean by the child nn.Modules ? How is a child normally defined ? Is there in the docs a definition of a child nn.Modules ?</p>

username: albanD
cooked: <p>What I call “child” here is just all the submodules that are registered into a given Module.<br>
submodules are registered automatically when set to an attribute of a Module directly <code>self.foo = my_module</code>.<br>
You can check the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Module">doc</a> for more details.</p>

username: FenryrMKIII
cooked: <p>Ok thanks !</p>
<p>That’s powerful <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>