Number of parameters in model summary is 0 for my custom model

username: Harshit_Pandey
cooked: <p>I have created a custom layer in which i have initialized a glorot uniform weight given input_shape = (1,3,299,299).</p>
<pre><code class="lang-auto">class CustomLayer(torch.nn.Module):
    def __init__(self, input_shape):
        super(CustomLayer,self).__init__()
        zeros = torch.zeros(input_shape)
        self.weights = torch.nn.Parameter(zeros)
        torch.nn.init.xavier_uniform_(self.weights)
    def forward(self, x):
        out= torch.tanh(self.weights)
        return out+x
        
class MimicAndFool(torch.nn.Module):
    def __init__(self,input_shape):
        super(MimicAndFool,self).__init__()
        self.custom = CustomLayer(input_shape)
    def forward(self,x):
        out = self.custom(x)
        #statement 2
        #statement 3...
        #statement n
        return out
</code></pre>
<p>After I print the summary</p>
<pre><code class="lang-auto">input_shape = (1,3,299,299)
maf = MimicAndFool(input_shape)
summary(maf,(3,299,299))
</code></pre>
<p>It says that the number of parameters for this custom layer is zero!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/1/7155c37c1ee2e539715fa5665d33552c0e68b956.png" data-download-href="https://discuss.pytorch.org/uploads/default/7155c37c1ee2e539715fa5665d33552c0e68b956" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/1/7155c37c1ee2e539715fa5665d33552c0e68b956.png" alt="image" data-base62-sha1="gaBHvVkMVXq8tpQOqsbD9pXJR6C" width="690" height="108" data-dominant-color="404040"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">807×127 3.59 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Please tell me what I’m missing out on . The number of parameters should have been = 3 * 299 * 299 = 268203</p>

username: ptrblck
cooked: <p>Your custom layer contains the <code>weigth</code> parameter printed by this code snippet:</p>
<pre><code class="lang-python">class CustomLayer(torch.nn.Module):
    def __init__(self, input_shape):
        super(CustomLayer,self).__init__()
        zeros = torch.zeros(input_shape)
        self.weights = torch.nn.Parameter(zeros)
        torch.nn.init.xavier_uniform_(self.weights)
    def forward(self, x):
        out= torch.tanh(self.weights)
        return out+x

class MimicAndFool(torch.nn.Module):
    def __init__(self,input_shape):
        super(MimicAndFool,self).__init__()
        self.custom = CustomLayer(input_shape)
    def forward(self,x):
        out = self.custom(x)
        #statement 2
        #statement 3...
        #statement n
        return out

layer = CustomLayer((1, 1))
print(dict(layer.named_parameters()))
&gt; {'weights': Parameter containing:
tensor([[-0.7979]], requires_grad=True)}

module = MimicAndFool((1, 1))
print(dict(module.named_parameters()))
&gt; {'custom.weights': Parameter containing:
tensor([[0.8589]], requires_grad=True)}
</code></pre>
<p>so I assume <code>torchsummary</code> might not return the right values for this custom module.</p>

username: Harshit_Pandey
cooked: <p>So I tried training the model, but the loss remains almost the same. The difference between the initial weights and final weights turn out to be zero, meaning that no training has actually happened for those weights. I’ve frozen all of the other layers except the custom layer.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba3ef83b36807fa469243d0836f87d05b352d7d4.png" data-download-href="https://discuss.pytorch.org/uploads/default/ba3ef83b36807fa469243d0836f87d05b352d7d4" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba3ef83b36807fa469243d0836f87d05b352d7d4_2_690x399.png" alt="image" data-base62-sha1="qzBIIVevyNr2WrVnykwahS0s2Ww" width="690" height="399" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba3ef83b36807fa469243d0836f87d05b352d7d4_2_690x399.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba3ef83b36807fa469243d0836f87d05b352d7d4_2_1035x598.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/a/ba3ef83b36807fa469243d0836f87d05b352d7d4_2_1380x798.png 2x" data-dominant-color="272827"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1459×845 53.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Could you check, if you get valid gradients in the custom module by calling</p>
<pre><code class="lang-python">print(model.custom.weights.grad)
</code></pre>
<p>after the <code>backward()</code> call?</p>

username: Harshit_Pandey
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/8/88985ed5a11648d40c0ddb9093bd6794cb758761.png" data-download-href="https://discuss.pytorch.org/uploads/default/88985ed5a11648d40c0ddb9093bd6794cb758761" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/8/88985ed5a11648d40c0ddb9093bd6794cb758761.png" alt="image" data-base62-sha1="juno454So4xOC2JylKmqvFdQPYZ" width="621" height="500" data-dominant-color="414141"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">852×685 22.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I do get grads for every iteration but they are very small it seems.</p>

username: Harshit_Pandey
cooked: <p>Oh i found out the reason why the final - initial weights were giving me a zero tensor <img src="https://discuss.pytorch.org/images/emoji/apple/man_facepalming.png?v=9" title=":man_facepalming:" class="emoji" alt=":man_facepalming:">. They were both pointing to the same object, when I printed them I got to know their difference.</p>

username: cutedora
cooked: <p>Hello, I have the same question with you.<br>
Could you tell me more about how you handle with the 0 trainable parameter in Pytorch summary problem?</p>