IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1) when Calling CrossEntropy loss

username: Harold_Ginger
cooked: <p>Hi, I am a beginner at DL, I am tring to design a patch-based net that using pretrained efficientnet as feature extractor.<br>
Currently, I divide one image into 35 patches and concatenate them to feed the model<br>
input shape is [35, 3, 300, 300](I tried batch_size &gt; 1 in Dataloader as someone told, but didn’t work for me)<br>
main part of my Dataset class</p>
<pre><code class="lang-auto">b = torch.zeros((len(patches)), 3, 300, 300)
            for i in range(len(patches)):
                b[i] = transforms.ToTensor()(patches[i])

            return b, label
</code></pre>
<p>main part of my net as follows</p>
<pre><code class="lang-auto">class Net():
    self.model = EfficientNet.from_pretrained('efficientnet-b3')
    ...
    def forward(self, x):
        x = self.model.extract_features(x)
        x = x * torch.sigmoid(x)
        x = F.adaptive_avg_pool2d(x, 1).squeeze(-1).squeeze(-1)
        x = view(-1)
        x = nn.Linear(35*1536, 1000, bias=True)
        x = nn.ReLU()
        x = nn.Dropout(p=0.5)
        x = nn.Linear(1000, num_classes, bias=True)
        return x
</code></pre>
<p>extract_features output shape is [35, 1536, 7, 7], so I flatten the tensor and feed it into a fc layer then get the result.<br>
But when I call CrossEntropy(net(input), label), it gave error as title, full track as below</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "models.py", line 114, in &lt;module&gt;
    train_loss_cur, train_acc_cur = train_fn(model, loader=train_loader)
  File "models.py", line 66, in train_fn
    loss = loss_fn(predictions, labels)
  File "/home/harold/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/harold/.local/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 932, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/harold/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 2317, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/harold/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1535, in log_softmax
    ret = input.log_softmax(dim)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
</code></pre>
<p>I searched a lot, but I just can’t figure out how to fix it, could someone be kind enough to tell me where did I go wrong, how to fix it. if detailed information is needed, I would post it ASAP.<br>
Thanks a lot advance, QAQ</p>

username: chetan_patil
cooked: <p>Looking at the output of <code>Net()</code>, it seems the output shape is <code>[N, num_classes]</code>.<br>
This means your <code>label</code> must be of size <code>[N]</code> such that each entries in <code>label</code> spans between 0 to <code>N-1</code>.<br>
Does the <code>label</code> satisfy the above requirements ?</p>

username: Harold_Ginger
cooked: <p>Thanks for your kind reply, I finally figure out what goes wrong.<br>
Since my dataset is consists of concatenated patches (shape[35, 3 300, 300]) , when load it with Dataloader, the shape become [batch_size, 35, 3, 300, 300], so I have to tranform it into [batch_size*35, 3, 300, 300], It seems l can’t just flatten the feature tensor, I have to keep the batch dim. Output of my old net  is [C], missing N.<br>
I change my forward code to</p>
<pre><code class="lang-auto">def forward(self, x):
        bs = x.size(0) 
        x = self.model.extract_features(x)
        x = x * torch.sigmoid(x)
        x = F.adaptive_avg_pool2d(x, 1).squeeze(-1).squeeze(-1)
        x = x.view(bs, -1)
        x = nn.Linear(35*1536, 1000, bias=True)
        x = nn.ReLU()
        x = nn.Dropout(p=0.5)
        x = nn.Linear(1000, num_classes, bias=True)
        return x
</code></pre>
<p>It works, but since it just transforms  tensor from [batch_size * patches, 1536]  to [batch_size, 1536*patches], which seems very ugly, I am wondering  is there any better solutions? Thanks again for your help!</p>