Dimension output of a model with criterion

username: aubin_thomas
cooked: <p>Hello,</p>
<p>I’m beginner with pytorch and with examples given in tutorials there is something I can’t understand.</p>
<p>With the Card classifier example, with this model:</p>
<pre><code class="lang-auto">class SimpleCardClassifier(nn.Module):
    def __init__(self, num_classes=53):
        super().__init__()
        self.base_model = timm.create_model('efficientnet_b0', pretrained=True)
        self.features = nn.Sequential(*list(self.base_model.children())[:-1])

        enet_out_size = 1280 # default output of efficient net
        # Make a classifier
        self.classifier = nn.Linear(enet_out_size, num_classes)

    def forward(self, x):
        x = self.features(x)
        output = self.classifier(x)
        return output
</code></pre>
<p>If I apply one data loader batch and apply a cross entropy criterion like this:</p>
<pre><code class="lang-auto">model = SimpleCardClassifier()
criterion = nn.CrossEntropyLoss()

for images, labels in dataloader:
    break

outputs = model(images)
print(outputs.shape)
print(labels.shape)
criterion(outputs, labels)
</code></pre>
<p>everything is fine:</p>
<pre><code class="lang-auto">torch.Size([64, 53])
torch.Size([64])
tensor(4.0855, grad_fn=&lt;NllLossBackward0&gt;)
</code></pre>
<p>But with this model</p>
<pre><code class="lang-auto">class CancerDataset(Dataset):
    def __init__(self, x_data, transform=None):
        self.data = x_data
        self.transform = transform
        self.nbcol = x_data.shape[1]

    def __len__(self):
        return len(self.data)

    def __getitem__(self, index):
        xcurrent = torch.tensor(self.data.iloc[index, 0:(self.nbcol-1)].values)
        xcurrent = xcurrent.to(torch.float32)
        ycurrent = torch.tensor(self.data.iloc[index, nbcol-1])

        if self.transform:
            xcurrent = self.transform(xcurrent)
        return xcurrent,ycurrent
</code></pre>
<p>by applying the same code :</p>
<pre><code class="lang-auto">model = CancerModel()
criterion = torch.nn.CrossEntropyLoss()
for features, labels in train_loader:
    break

outputs = model(features)
print(outputs.shape)
print(labels.shape)
criterion(outputs, labels.float())
</code></pre>
<p>the dimensions of the outputs follow the same way (except one dimensional output)</p>
<pre><code class="lang-auto">torch.Size([64, 1])
torch.Size([64])
</code></pre>
<p>But I have a error “expected scalar type Long but found Float” because of Criterion, that is not raised if I change the call of loss like this:</p>
<p><code>criterion(outputs, labels.view(-1,1).float())</code></p>
<p>that changes the dimension of labels as torch.Size([64,1])</p>
<p>I don’t understand at all what’s going on</p>

username: ptrblck
cooked: <p>The error is raised in <code>nn.CrossEntropyLoss</code> as it either expects:</p>
<ul>
<li><code>LongTensor</code>s as the target containing the class indices in the range <code>[0, nb_classes-1]</code> and the shape <code>[batch_size, *]</code> (without the <code>nb_classes</code> dimension the model output would have) (which seems to be the case in your first approach)</li>
<li>or <code>FloatTensor</code>s as the target containing “soft labels” in the range <code>[0, 1]</code> having the same shape as the model output (which seems to be the case in your second approach).</li>
</ul>
<p>Instead of reshaping the target in the second approach, transforming it to an integer type via <code>labels.long()</code> should also work.</p>