Pytorch Error IndexError; dimension specified as 0 but tensor has no dimensions

username: Dmitriy
cooked: <p>Hello!<br>
I have dataset where: data 23761 x 13 , labels 23761 x 1 loaded by DataLoader<br>
My net:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(13, 13)
        self.fc2 = nn.Linear(13, 2)

    def forward(self,x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return F.logsigmoid(x)

net = Net()
criterion = nn.CrossEntropyLoss()

carset = carDataset()
train_loader = torch.utils.data.DataLoader(carset, batch_size = 10, shuffle = True )

for epoch in range(epochs):
   running_loss = 0
   for features, labels in train_loader:
       output = net(features)
       labels = labels.squeeze()
       loss = criterion(output, labels)
       optimizer.zero_grad()
       loss.backward()
       optimizer.step()

       running_loss += loss.item()

   print(loss.item())
</code></pre>
<p>When I put <code>batch_size = 10</code> or <code> batch_size = 20</code> it throws:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "nn.py", line 75, in &lt;module&gt;
    loss = criterion(output, labels)
 
    if input.size(0) != target.size(0):
IndexError: dimension specified as 0 but tensor has no dimensions
</code></pre>
<p>When I make in 64 it works normally and I have results.</p>
<p>My labels have 2 classes: 0 or 1.<br>
Whats wrong with it ?</p>

username: CedricPicron
cooked: <p>Hi <a class="mention" href="/u/dmitriy">@Dmitriy</a>. The reason it fails is because the last batch of your epoch will be of size 1 when using  <code>batch_size = 10</code> or <code>batch_size = 20</code>, as your dataset length is 23761. Because of this, <code>labels.size() = torch.Size([1, 1])</code> will result in a size of <code>torch.Size([])</code> after</p>
<aside class="quote no-group" data-username="Dmitriy" data-post="1" data-topic="96373">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmitriy/48/19414_2.png" class="avatar"> Dmitriy:</div>
<blockquote>
<p><code>labels = labels.squeeze()</code></p>
</blockquote>
</aside>
<p>giving the error you mention. To solve this replace the line by <code>labels = labels.squeeze(1)</code> to avoid squeezing the batch dimension, or set <code>drop_last=True</code> in your Dataloader.</p>
<p>Note moreover that <code>nn.CrossEntropyLoss()</code> already performs <code>nn.LogSoftmax()</code> (see <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss" class="inline-onebox" rel="noopener nofollow ugc">CrossEntropyLoss — PyTorch 2.1 documentation</a>). Doublecheck if</p>
<aside class="quote no-group" data-username="Dmitriy" data-post="1" data-topic="96373">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmitriy/48/19414_2.png" class="avatar"> Dmitriy:</div>
<blockquote>
<p><code>return F.logsigmoid(x)</code></p>
</blockquote>
</aside>
<p>is desired. Hope this helps!</p>

username: Dmitriy
cooked: <p>Well, it is going to be logical!<br>
Thank you for your supporting. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>
<p>P.S. About <code>nn.CrossEntropyLoss()</code> :I am a newbie in PyTorch and not really researching all functions right.<br>
It seems I need use <code>criterion = nn.NLLLoss()</code> for reducing excess work ?</p>

username: CedricPicron
cooked: <aside class="quote no-group" data-username="Dmitriy" data-post="3" data-topic="96373">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmitriy/48/19414_2.png" class="avatar"> Dmitriy:</div>
<blockquote>
<p>It seems I need use <code>criterion = nn.NLLLoss()</code> for reducing excess work ?</p>
</blockquote>
</aside>
<p>I believe using <code>F.logsigmoid(x)</code> might be undesired as your (log) probabilities will be independent for both classes (one class might have probability 0.5 and the other 0.7).  Your model might still train well, but the outputted probabilities will not add up to one. For simplicity, I suggest to use</p>
<pre data-code-wrap="python"><code class="lang-python">return x
</code></pre>
<p>instead of</p>
<pre data-code-wrap="python"><code class="lang-python">return F.logsigmoid(x)
</code></pre>
<p>and use <code>nn.CrossEntropyLoss()</code> which will take care of both <code>nn.LogSoftmax()</code> and <code>nn.NLLLoss()</code>. Using <code>nn.LogSoftmax()</code> will make sure your probabilities across classes add up to one.</p>