Size of input tensor in a Linear Layer

username: Shell1500
cooked: <p>I’ve started to work with pytorch a  few weeks ago.(No prior knowledge of ML)</p>
<p>I want to biuld a image classifier that detects wheather an image is a cat or a dog.</p>
<p>I have data in the form of:</p>
<pre><code class="lang-auto">tensor([[1.6886e-06, 4.2819e-06, 1.3871e-06,  ..., 1.4353e-05, 1.4173e-05,
         1.3931e-05],
        [1.5680e-06, 4.9453e-06, 1.2062e-06,  ..., 1.5017e-05, 1.4896e-05,
         1.4173e-05],
        [1.3871e-06, 5.3072e-06, 2.5330e-06,  ..., 1.4052e-05, 1.3992e-05,
         1.3992e-05],
        ...,
        [5.8499e-06, 5.9706e-06, 5.2469e-06,  ..., 2.4727e-06, 1.8093e-06,
         2.5933e-06],
        [5.0056e-06, 5.1262e-06, 5.3072e-06,  ..., 3.6185e-06, 1.4474e-06,
         1.8696e-06],
        [4.8247e-06, 5.1865e-06, 5.2469e-06,  ..., 4.1613e-06, 3.2567e-06,
         1.8093e-06]])
tensor([[0., 1.],
        [0., 1.],
        [1., 0.],
        ...,
        [0., 1.],
        [0., 1.],
        [0., 1.]])
</code></pre>
<p>The image is a 50 by 50 gray scale and the output is a 1D tensor.</p>
<p>This is my Network Class:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(50*50, 64)
        self.fc2 = nn.Linear(64, 64)
        self.fc3 = nn.Linear(64, 64)
        self.fc4 = nn.Linear(64, 2)
        
    def forward(self, x):

        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = self.fc4(x)
        
        return F.log_softmax(x, dim=1)  
        

net = Net()
</code></pre>
<p>This is my training loop using <code>optim.Adam</code>:</p>
<pre><code class="lang-auto">EPOCHS=6

for epoch in range(EPOCHS):
    for key in range(24946):
        X = data[key]
        y =out[key]
        net.zero_grad()
        output = net(X)
        loss = F.nll_loss(output, y)
        loss.backward()
        optimizer.step()
    print(loss)
</code></pre>
<p>However i get the following error:<br>
<code>RuntimeError: size mismatch, m1: [1 x 50], m2: [2500 x 64]</code></p>
<p>I believe this is due to the input to my first layer.<br>
Can someone please explain how I would input each image to my Network?</p>

username: KFrank
cooked: <p>Hello Shell!</p>
<aside class="quote no-group quote-modified" data-username="Shell1500" data-post="1" data-topic="76589" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shell1500/48/23958_2.png" class="avatar"> Shell1500:</div>
<blockquote>
<p>I have data in the form of:</p>
<pre><code class="lang-auto">tensor([[1.6886e-06, 4.2819e-06, 1.3871e-06,  ..., 1.4353e-05, 1.4173e-05,
         1.3931e-05],
...
</code></pre>
</blockquote>
</aside>
<p>I am <em>guessing</em> that this tensor is the <code>X</code> you input to your<br>
network in <code>net(X)</code> and that is has shape <code>[50, 50]</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>The image is a 50 by 50 gray scale</p>
<p>This is my Network Class:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(50*50, 64)
...
</code></pre>
<p>…</p>
<p>However i get the following error:<br>
<code>RuntimeError: size mismatch, m1: [1 x 50], m2: [2500 x 64]</code></p>
</blockquote>
</aside>
<p>The problem is that pytorch networks (models) <em>always</em><br>
work with batches of input samples (even if you don’t<br>
want them to –  if you want to pass a single sample to a<br>
network, you have to wrap it in a batch of batch-size 1).</p>
<p>So (according to my theory) pytorch is interpreting your<br>
shape <code>[50, 50]</code> input tensor as a batch of 50 input samples,<br>
where each sample has shape <code>[50]</code>.  But your first <code>Linear</code><br>
layer is expecting an input of shape <code>[2500]</code>, hence the<br>
“size mismatch” error (50 != 2500).</p>
<p>You can print out <code>X.shape</code> (and <code>y.shape</code>, for that matter)<br>
to see if this is what is going on.</p>
<p>(Note, <code>Linear (50*50, 64)</code> is <em>exactly</em> the same as<br>
<code>Linear (2500, 64)</code>.  The fact that you write the number<br>
<code>2500</code> as <code>50*50</code> does not somehow tell the layer to accept<br>
an input of  shape <code>[50, 50]</code>.)</p>
<p>You would first need to <code>flatten()</code> your input tensor <code>X</code> to<br>
give it shape <code>[2500]</code>, and then <code>unsqueeze()</code> it to turn in<br>
into a batch (of batch-size 1 containing only one sample)<br>
with shape <code>[1, 2500]</code>.</p>
<p>Thus:<br>
<code>X = torch.unsqueeze (torch.flatten (X), dim = 0)</code></p>
<p>Good luck.</p>
<p>K. Frank</p>