Some general questions about pytorch model

username: krishna511
cooked: <ol>
<li>why this <code>*</code> is used with the <code>input</code> forward function?</li>
</ol>
<pre><code class="lang-auto">def forward(self, *input):
</code></pre>
<ol start="2">
<li>If the shape of the output of a model is 32*4. I am not getting why this <code>unsqueeze(1)</code> here:</li>
</ol>
<pre><code class="lang-auto"> out = model(x.unsqueeze(1))
</code></pre>
<ol start="3">
<li>This question is really silly. But please forgive me.<br>
If x is 8*64 and it is passed to a fully connected layer as:</li>
</ol>
<pre><code class="lang-auto"> self.fc = nn.Linear(in_features=8, out_features=4) 
 x = self.fc(x)
</code></pre>
<p>Then how many weight matrices will be there <code>256*512</code> and then <code>4*256</code>, Here 512 is from <code>8*46</code> am I right ?</p>
<p>Regards</p>

username: krishna511
cooked: <p>I got the answer for the second: It will add a dimension at first position i.e. <code>32*1*4</code>.</p>

username: ptrblck
cooked: <ol>
<li>the <code>*</code> in front if the <code>input</code> allows for a variable number of arguments as seen here:</li>
</ol>
<pre><code class="lang-python">def myfun(*inputs):
    print(inputs)

# single input
myfun(torch.tensor(1))
&gt; (tensor(1),)

# list input
myfun([torch.tensor(1), torch.tensor(1)])
&gt; ([tensor(1), tensor(1)],)

# multiple inputs
myfun(torch.tensor(1), torch.tensor(1))
&gt; (tensor(1), tensor(1))
</code></pre>
<ol start="3">
<li>The linear layer will have a <code>weight</code> parameter in the shape <code>[4, 8]</code> and an input in the shape <code>[8, 64]</code> will raise a shape mismatch error, since <code>8</code> input features are expected.</li>
</ol>

username: krishna511
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you sir. Plz, comment on whether I am correct with the second answer. Also If I am printing <code>x.shape</code> in this forward function,</p>
<pre><code class="lang-auto">def forward(self, *input):
        #print(input[0].shape)
        xa = self.conv1a(input[0])
        
        xa = self.bn1a(xa)

        xa = F.relu(xa)
        xb = self.conv1b(input[0])
        xb = self.bn1b(xb)

        xb = F.relu(xb)
        x = torch.cat((xa, xb), 1)
        x = self.conv2(x)
        x = self.bn2(x)

        x = F.relu(x)
        x = self.maxp(x)
        ..
..

        x = self.conv5(x)
        x = self.bn5(x)

        x = F.relu(x)
        print(x.shape)
</code></pre>
<p>I am getting something like this</p>
<pre><code class="lang-auto">torch.Size([7, 80, 6, 15])
torch.Size([5, 80, 6, 15])
torch.Size([2, 80, 6, 15])
torch.Size([6, 80, 6, 15])
torch.Size([5, 80, 6, 15])
torch.Size([12, 80, 6, 15])
torch.Size([8, 80, 6, 15])
</code></pre>
<p>Sir I know the shape is printed as many times as the <code>total_samples</code>, Sir <a class="mention" href="/u/ptrblck">@ptrblck</a> Here 80 is the output channels, <code>6*15</code> is the shape of each channel output. But I am not getting why this o dimension is changing every time.<br>
Regards</p>

username: ptrblck
cooked: <p>Assuming <code>dim0</code> represents the batch dimension, the activation shapes seem wrong, since the number of samples would change at these points.<br>
Could you post an executable code snippet, which would reproduce this behavior?</p>

username: krishna511
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> sure sir. I will in some time. What is <code>activation shape</code>?</p>

username: ptrblck
cooked: <p>By activation I meant the outputs of any layer, i.e. the tensors you pass from one layer to the next in the <code>forward</code> method:</p>
<pre><code class="lang-python">def forward(self, input):
    activation = self.layer(input)
    activation = self.other_layer(activation)
</code></pre>

username: krishna511
cooked: <p>Oh okay sir!<br>
But I think the shapes are okay otherwise, the model would have thrown error right?</p>