Multi Label Classification Evaluation and Training Loop

username: ssalome
cooked: <p>Hi<br>
I’m currently doing a multi label classification problem<br>
As far as I know using BCELogitsLoss() function is used as a loss function for such type of problems</p>
<p>I have images and one hot vectors and the image ids as input<br>
Example<br>
imagetensor =&gt;  tensor([[[-1.2406, -1.6744, -1.8826,  …, -1.9694, -1.9347, -1.8826],<br>
[-1.8306, -1.9694, -1.9867,  …, -1.9867, -1.9867, -1.9867],<br>
[-1.9867, -1.9867, -1.9867,  …, -1.9867, -1.9867, -1.9867],<br>
…,<br>
[-1.9867, -1.9867, -1.9867,  …, -1.9867, -1.9867, -1.9867],<br>
[-1.9867, -1.9867, -1.9867,  …, -1.9867, -1.9867, -1.9867],<br>
[-1.7264, -1.8653, -1.8826,  …, -1.9867, -1.9867, -1.9867]]])<br>
one hot vector =&gt; [1. 0. 1. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]</p>
<p>This is the evaluation function</p>
<pre><code class="lang-auto">loss_fn = nn.BCEWithLogitsLoss()
opt = optim.SGD(resnet.parameters(), lr = 0.01)
</code></pre>
<pre><code class="lang-auto">def evaluation(dataloader, model):
  # model added as argument - 1 change from lenet
  total, correct = 0,0
  for data in dataloader:
    inputs, labels,ids = data
    inputs, labels, ids = inputs.to(device), labels.to(device), ids.to(device)
    # inputs are put in the gpu
    outputs = model(inputs)
    print(outputs)
    # from net to model - 2 change from lenet
    _, pred = torch.max(outputs.data,1)
    print(pred)
    total += labels.size(0)
    correct +=(pred==labels).sum().item()
  return 100 * correct/total
</code></pre>
<p>Training loop</p>
<pre><code class="lang-auto">loss_epoch_arr = []
max_epochs = 10
min_loss = 1000

n_iters = np.ceil(len(trainset)/batch_size)

for epoch in range(max_epochs):
    for i, data in enumerate(trainloader, 0):

        inputs, labels , imgindex = data
        inputs, labels, imgindex  = inputs.to(device), labels.to(device), imgindex.to(device)

        #print(inputs.shape)
        opt.zero_grad()

        outputs = resnet(inputs)
        loss = loss_fn(outputs, labels)
        loss.backward()
        opt.step()

        if min_loss&gt; loss.item():
            min_loss = loss.item()
            best_model = copy.deepcopy(resnet.state_dict())
            print('Min loss %0.2f' % min_loss)

        if i%100 == 0:
            print('Iteration : %d/%d, Loss : %0.2f' % (i, n_iters, loss.item()))
    
        del inputs, labels, imgindex, outputs
        torch.cuda.empty_cache()

    loss_epoch_arr.append(loss.item())

    print('Epoch: %d/%d, Train acc : %0.2f, Test acc : %0.2f' % (epoch, max_epochs, evaluation(trainloader,resnet), evaluation(testloader,resnet)) )   
plt.plot(loss_epoch_arr)
plt.show()
</code></pre>
<p>I’m using resnet pretrained with imagenet weights and have modified the first layer to accept grey scale images and the number of output classes to 15 in the last layer.</p>
<p>On running the training loop, the print statement calling the evaluation(trainloader,resnet) causes an error, on running with the following runtime settings<br>
<strong>CPU -&gt; RuntimeError: The size of tensor a (250) must match the size of tensor b (15) at non-singleton dimension 1, the batch size I’m using is 250, 15 is number of classes</strong></p>
<p><strong>GPU -&gt;     9 **<br>
**     10         inputs, labels , imgindex = data</strong><br>
<strong>—&gt; 11         inputs, labels, imgindex  = inputs.to(device), labels.to(device), imgindex.to(device)</strong></p>
<p><strong>AttributeError: ‘tuple’ object has no attribute 'to’</strong><br>
<strong>I</strong>’m new to multi label classification…So any help would be appreciated. <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: KFrank
cooked: <p>Hello Salome!</p>
<p>It looks as if you might be mixing together things for multi-label<br>
(multi-class) and single-label (multi-class) problems.</p>
<aside class="quote no-group quote-modified" data-username="ssalome" data-post="1" data-topic="81553" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/43a26b/48.png" class="avatar"> ssalome:</div>
<blockquote>
<p>I’m currently doing a multi label classification problem<br>
…<br>
one hot vectors</p>
</blockquote>
</aside>
<p>One-hot vectors are typically be used to encode the labels for<br>
a single-label, multi-class problem.  (But you normally use integer<br>
class labels instead of one-hot encoded labels.)</p>
<aside class="quote no-group">
<blockquote>
<p>one hot vector =&gt; [1. 0. 1. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]</p>
</blockquote>
</aside>
<p>Note, this is not a one-hot vector because multiple elements have<br>
the value <code>1</code>, not just one.  (Also, as written, it has no comma<br>
separators.)</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">    _, pred = torch.max(outputs.data,1)
...
    correct +=(pred==labels).sum().item()
</code></pre>
</blockquote>
</aside>
<p>This is how you would compute the number of correct predictions is<br>
a single-label, multi-class problem.  Note that <code>pred</code>, the second<br>
element of the tuple returned by <code>torch.max()</code> is <code>argmax()</code>, that is,<br>
the index of the largest value (along dimension 1) in <code>output</code>.  This<br>
would be a single class label for your sample, not a set of multiple<br>
active class labels.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">        inputs, labels , imgindex = data
...
        outputs = resnet(inputs)
</code></pre>
<p>the number of output classes to 15 in the last layer.</p>
</blockquote>
</aside>
<p>Could you print out the <code>shape</code> and <code>type()</code> of <code>inputs</code>, <code>labels</code>,<br>
<code>imgindex</code>, and <code>outputs</code>?  Could you also show the code for the last<br>
<code>Linear</code> layer of your model and any activations that follow it?</p>
<p>Best.</p>
<p>K. Frank</p>

username: ssalome
cooked: <p>Hi K. Frank,<br>
<strong>I want to do a multi-label (multiple class) problem</strong><br>
<strong>one sample can belong to more than 1 class</strong><br>
<strong>so this means [1. 0. 1. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0.] that the sample belongs to 3 classes</strong></p>
<p>(tensor([[[-1.9000, -1.9000, -1.9000,  …, -1.9000, -1.9000, -1.9000],<br>
[-1.9000, -1.9000, -1.9000,  …, -1.8826, -1.8826, -1.8826],<br>
[-1.9000, -1.9000, -1.9000,  …, -1.8826, -1.8826, -1.8826],<br>
…,<br>
[ 0.2864,  0.5467,  0.6334,  …, -0.5118, -0.4771, -0.5465],<br>
[ 0.2864,  0.5293,  0.6508,  …, -0.5292, -0.4597, -0.5118],<br>
[ 0.0955,  0.2864,  0.4079,  …, -0.6506, -0.5986, -0.6159]]]), array([0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0.],<br>
dtype=float32), ‘00001250_006.png’)</p>
<p>this is a tuple(input_img tensor, output_vector, image_id)</p>
<p>Image type and shape =&gt; &lt;class ‘torch.Tensor’&gt; torch.Size([1, 224, 224])<br>
Output type and shape =&gt;&lt;class ‘numpy.ndarray’&gt; (15,)<br>
Image id/imgindex(name of the image file) type=&gt;&lt;class ‘str’&gt;</p>
<p>(1): BasicBlock(<br>
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
<strong>(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))</strong><br>
<strong>(fc): Linear(in_features=512, out_features=15, bias=True)</strong><br>
)</p>
<p>Last layers of Resnet</p>

username: ssalome
cooked: <p>And I also want to get the probabilities for the predicted classes like 70% it belongs to class1, 50% it belongs to class 5, etc for a given sample image.</p>

username: KFrank
cooked: <p>Hi Salome!</p>
<aside class="quote no-group quote-modified" data-username="ssalome" data-post="3" data-topic="81553" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/43a26b/48.png" class="avatar"> ssalome:</div>
<blockquote>
<p>(tensor([[[-1.9000, -1.9000, -1.9000,  …, -1.9000, -1.9000, -1.9000],<br>
…<br>
[ 0.0955,  0.2864,  0.4079,  …, -0.6506, -0.5986, -0.6159]]]), array([0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0.],<br>
dtype=float32), ‘00001250_006.png’)</p>
<p>this is a tuple(input_img tensor, output_vector, image_id)</p>
</blockquote>
</aside>
<p>Well, this is confusing.  In the code you posted, you had the variables<br>
<code>inputs</code>, <code>labels</code>, and <code>imgindex</code>.  But now you seem to be calling<br>
them “input_img tensor, output_vector, image_id”.  I will <em>guess</em> that<br>
these are the same things.</p>
<aside class="quote no-group">
<blockquote>
<p>Image type and shape =&gt; &lt;class ‘torch.Tensor’&gt; torch.Size([1, 224, 224])<br>
Output type and shape =&gt;&lt;class ‘numpy.ndarray’&gt; (15,)<br>
Image id/imgindex(name of the image file) type=&gt;&lt;class ‘str’&gt;</p>
</blockquote>
</aside>
<p>Now I’m <em>guessing</em> that “Image” is what you just called “input_img<br>
tensor", “Output” is “output_vector”, and “Image id/imgindex(name<br>
of the image file)”  is “image_id”.</p>
<p>(And I’m also guessing that you haven’t given the <code>shape</code> and <code>type()</code><br>
of the variable <code>outputs</code> in your code, that is, the object returned by<br>
your <code>model</code>.)</p>
<p>So it looks like what your <em>code</em> calls <code>inputs</code> is a single 224x224 image<br>
with a leading <em>singleton</em> (<code>size = 1</code>) dimension, and that what your<br>
code calls <code>labels</code> is a single numpy array of (floating-point) binary<br>
labels, one for each of your fifteen classes.  Note, I don’t see your<br>
batch size of 250 anywhere.</p>
<p>Your <code>inputs</code> tensor should be a <em>batch</em> of <code>250</code> images.  It should<br>
most likely be a tensor of <code>shape</code> <code>[250, 224, 244]</code>, unless your<br>
<code>model</code> is expecting its input images to carry a “channel” dimension<br>
of <code>nChannel = 1</code>, in which case your input tensor should have<br>
<code>shape</code> <code>[250, 1, 224, 244]</code>.</p>
<p>Your <code>labels</code> tensor should be  a pytorch tensor (not a numpy array),<br>
and should be a  <em>batch</em> of sets of (floating-point) binary class labels,<br>
one for each of your 15 classes, and therefore should have <code>shape</code><br>
<code>[250, 15]</code>.</p>
<aside class="quote no-group">
<blockquote>
<p><strong>(fc): Linear(in_features=512, out_features=15, bias=True)</strong></p>
</blockquote>
</aside>
<p>Although you didn’t provide the information for what your <code>outputs</code><br>
variable is, logically, and based on the last layer of your <code>model</code>, it<br>
should have <code>shape</code> <code>[batch size, nClass]</code>, that is, <code>[250, 15]</code>.<br>
It will be your <code>model</code>’s predicted raw-score <em>logits</em> for each of your<br>
250 samples being in each of your 15 classes.</p>
<p>This is what your data should look like.  But it looks like the <code>shape</code> of<br>
your <code>model</code> <code>outputs</code> and the shape of your <code>labels</code> aren’t matching<br>
up properly, presumably somewhere in your <code>evaluation()</code> function.</p>
<p>Could you call something like</p>
<pre data-code-wrap="python"><code class="lang-python">trainAcc = evaluation (trainloader, resnet)
print (trainAcc)
testAcc = evaluation (testloader, resnet)
print (testAcc)
</code></pre>
<p><em>before</em> you run any training?  (And call <code>evaluation()</code> in two separate<br>
lines of code, as above.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: ssalome
cooked: <p>Hi K. Frank,</p>
<p>print(‘Epoch: %d/%d, Train acc : %0.2f, Test acc : %0.2f’ % (epoch, max_epochs, evaluation(trainloader,resnet), evaluation(testloader,resnet)) )</p>
<p>when i run this, i get the following output</p>
<p>I’m printing  outputs = model(inputs)<br>
<strong>print(outputs)</strong><br>
_, pred = torch.max(outputs.data,15)<br>
<strong>print(pred)</strong> within the evaluation function</p>
<h2>tensor([[-1.3976, -1.9027, -0.8711,  …, -1.4458, -1.2116, -0.8598],<br>
[-0.3187, -0.8117, -1.0978,  …, -2.2132, -1.3070, -0.9956],<br>
[-0.8397, -1.6915, -0.6634,  …, -1.8321, -1.0346, -1.2636],<br>
…,<br>
[-1.4526, -1.1693, -1.0159,  …, -1.3135, -1.1257, -0.9665],<br>
[-1.3092, -1.4912, -1.3867,  …, -2.0305, -1.1680, -1.3192],<br>
[-0.5745, -1.1265, -0.9143,  …, -2.1783, -0.6507, -1.4715]],<br>
grad_fn=)<br>
tensor([ 3,  3,  5,  3,  3,  3,  3,  7,  3,  3,  3,  3,  3,  3,  7,  3,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,<br>
13,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  9,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  2,  3,  3,  5,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  8,  5,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  8,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  5,  3,  5,  3,  3,<br>
3,  3,  3,  3,  3,  3,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  8,  3,  3,  3,  3,  3,  5,  2,  3,  3,<br>
3,  3,  3,  3,  5,  5,  3,  2,  3,  3,  3,  3,  3,  9,  3,  3,  3,  2,<br>
3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3, 13,  5,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  3,  8,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,<br>
3,  3,  3,  3,  3,  3,  3,  3,  4,  3,  3,  3,  3,  3,  3,  3])</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 print(‘Epoch: %d/%d, Train acc : %0.2f, Test acc : %0.2f’ % (epoch, max_epochs, evaluation(trainloader,resnet), evaluation(testloader,resnet)) )</p>
<p>1 frames<br>
 in evaluation(dataloader, model)<br>
12     print(pred)<br>
13     total += labels.size(0)<br>
—&gt; 14     correct +=(pred==labels).sum().item()<br>
15   return 100 * correct/total<br>
16 <span class="hashtag">#preds</span> = torch.tensor([])</p>
<p>/usr/local/lib/python3.6/dist-packages/torch/tensor.py in wrapped(*args, **kwargs)<br>
26     def wrapped(*args, **kwargs):<br>
27         try:<br>
—&gt; 28             return f(*args, **kwargs)<br>
29         except TypeError:<br>
30             return NotImplemented</p>
<p>RuntimeError: The size of tensor a (250) must match the size of tensor b (15) at non-singleton dimension 1</p>
<p>The dataset is imbalanced hence it predicts ‘3’ in most of the cases as there are more samples of class corresponding to the index ‘3’</p>

username: ssalome
cooked: <p>I think from the output that I get it is of shape [250, 1] and not [250, 15]. It should be [250, 15]. I’m not understanding where I have to make the change within the training loop or the evaluation function.</p>

username: ssalome
cooked: <aside class="quote no-group" data-username="KFrank" data-post="5" data-topic="81553">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Well, this is confusing. In the code you posted, you had the variables<br>
<code>inputs</code> , <code>labels</code> , and <code>imgindex</code> . But now you seem to be calling<br>
them “input_img tensor, output_vector, image_id”. I will <em>guess</em> that<br>
these are the same things.</p>
</blockquote>
</aside>
<p>Yes it is the same thing</p>

username: ssalome
cooked: <aside class="quote no-group" data-username="KFrank" data-post="5" data-topic="81553">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Image type and shape =&gt; &lt;class ‘torch.Tensor’&gt; torch.Size([1, 224, 224])<br>
Output type and shape =&gt;&lt;class ‘numpy.ndarray’&gt; (15,)<br>
Image id/imgindex(name of the image file) type=&gt;&lt;class ‘str’&gt;</p>
</blockquote>
</aside>
<p>Ya this is just for a single sample and not for the batch</p>

username: ssalome
cooked: <aside class="quote no-group quote-modified" data-username="KFrank" data-post="5" data-topic="81553">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Your <code>labels</code> tensor should be a pytorch tensor (not a numpy array),<br>
and should be a <em>batch</em> of sets of (floating-point) binary class labels,<br>
one for each of your 15 classes, and therefore should have <code>shape</code><br>
<code>[250, 15]</code> .</p>
<aside class="quote no-group">
<blockquote>
<p><strong>(fc): Linear(in_features=512, out_features=15, bias=True)</strong></p>
</blockquote>
</aside>
</blockquote>
</aside>
<p>So do I need to change the last year out_features to  [250, 15] or something else?</p>

username: KFrank
cooked: <p>Hello Salome!</p>
<p>It’s very hard to know what you are doing.  You post some code, and<br>
then you post some data that might or might not be related, and the<br>
data you post has been renamed and edited.</p>
<p>I’ve asked you to post some specific values.  You post things that might<br>
or might not be related, but usually differ at least somewhat from what<br>
I’ve asked you to post.</p>
<p>You don’t reflect on things I’ve tried to explain.</p>
<p>For example, in my first reply I noted that using:</p>
<pre data-code-wrap="python"><code class="lang-python">    _, pred = torch.max(outputs.data,1)
</code></pre>
<p>to convert your <code>outputs</code> to <code>pred</code> is appropriate for “a single-label,<br>
multi-class problem.”  You reiterate that yours is a multi-label problem.<br>
But the tensor you just posted for <code>pred</code> looks like the result of the<br>
<code>torch.max()</code> statement.  But I don’t know, because I can only guess<br>
what code produced the <code>pred</code> result you just posted.</p>
<aside class="quote no-group quote-modified" data-username="ssalome" data-post="10" data-topic="81553" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/43a26b/48.png" class="avatar"> ssalome:</div>
<blockquote>
<p><strong>(fc): Linear(in_features=512, out_features=15, bias=True)</strong></p>
<p>So I’ll have to change out_features to accept shape (250, 15)?</p>
</blockquote>
</aside>
<p>No.</p>
<p>It is important to understand that you write pytorch models from the<br>
perspective of processing a single sample.  But the pytorch framework<br>
actually processes samples in batches.</p>
<p>So, for example, if you write your model to take as input a tensor of<br>
<code>shape</code> <code>[244, 244]</code> (that is a single sample), and produce as output<br>
a tensor of <code>shape</code> <code>[15]</code> (that are the fifteen binary class labels it<br>
predicts for that single sample), then when you run your model, you<br>
have to pass in a <em>batch</em> of samples of <code>shape</code> <code>[nBatch, 244, 244]</code>,<br>
and the model will return a <em>batch</em> of predicted class labels of <code>shape</code><br>
<code>[nBatch, 15]</code>.</p>
<p>Note, the value of <code>nBatch</code> is not wired into your model.  You can pass<br>
in a batch of 250 (<code>[250, 244, 244]</code>), getting out a batch of 250<br>
(<code>[250, 15]</code>), and then, without changing the model pass in a batch<br>
of 1 (<code>[1, 244, 244]</code> → <code>[1, 15]</code>).  (Also note, that if you want to apply<br>
your model to a single sample, that sample <em>has</em> to be wrapped in<br>
a batch of 1.  That is, you can’t pass in <code>[244, 244]</code>; you have to pass<br>
in <code>[1, 244, 244]</code>).</p>
<p>So, to repeat, your batch size of 250 does not appear in the code for<br>
your model.  And even though the last linear layer looks as if it outputs<br>
a tensor of shape <code>[15]</code> (<code>out_features=15</code>), because pytorch models<br>
always work with batches, your model will output a tensor of <code>shape</code><br>
<code>[nBatch, 15]</code>, where <code>nBatch</code> is the size of the batch you passed<br>
into your model.</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: ssalome
cooked: <aside class="quote no-group quote-modified" data-username="KFrank" data-post="11" data-topic="81553">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>Note, the value of <code>nBatch</code> is not wired into your model. You can pass<br>
in a batch of 250 ( <code>[250, 244, 244]</code> ), getting out a batch of 250<br>
( <code>[250, 15]</code> ), and then, without changing the model pass in a batch<br>
of 1 ( <code>[1, 244, 244]</code> → <code>[1, 15]</code> ). (Also note, that if you want to apply<br>
your model to a single sample, that sample <em>has</em> to be wrapped in<br>
a batch of 1. That is, you can’t pass in <code>[244, 244]</code> ; you have to pass<br>
in <code>[1, 244, 244]</code> ).</p>
<p>So, to repeat, your batch size of 250 does not appear in the code for<br>
your model. And even though the last linear layer looks as if it outputs<br>
a tensor of shape <code>[15]</code> ( <code>out_features=15</code> ), because pytorch models<br>
always work with batches, your model will output a tensor of <code>shape</code><br>
<code>[nBatch, 15]</code> , where <code>nBatch</code> is the size of the batch you passed<br>
into your model.</p>
</blockquote>
</aside>
<p>inputs.shape &amp; type  torch.Size([64, 1, 224, 224]) &lt;class ‘torch.Tensor’&gt;<br>
labels.shape &amp; type torch.Size([64, 15]) &lt;class ‘torch.Tensor’&gt;<br>
imgindex &amp; type (‘00001151_002.png’, ‘00000583_010.png’, ‘00000500_000.png’, ‘00000271_000.png’, …, ‘00000032_031.png’, ‘00000913_000.png’, ‘00001075_024.png’)<br>
outputs, shape &amp; type tensor([[-2.1863, -2.8136, -2.0755,  0.1303, -2.4541, -2.1647, -3.0948, -2.5282,<br>
-1.9832, -2.4096, -2.6687, -2.7546, -2.1767, -2.6486, -3.0040],<br>
[-2.3346, -1.7464, -1.3218, -0.2523, -1.8560, -0.8745, -1.6130, -1.5004,<br>
-1.5930, -1.9782, -2.1910, -2.3470, -1.8963, -2.1992, -1.6291],<br>
[-2.0328, -2.0936, -1.9872,  0.6333, -2.5061, -1.2418, -2.6371, -3.0564,<br>
-2.0829, -2.0732, -2.0986, -2.5197, -2.8750, -2.4518, -2.9555],<br>
…<br>
[-2.5923, -2.6953, -1.5379,  0.2798, -2.7280, -1.5506, -2.4956, -1.8402,<br>
-2.2393, -2.3018, -2.3598, -1.9540, -3.1476, -2.3570, -1.6612]],<br>
device=‘cuda:0’, grad_fn=) <strong>torch.Size([64, 15])</strong> &lt;built-in method type of Tensor object at 0x7feedc76ec18&gt;</p>
<p>Hi K. Frank,<br>
The value of nBatch is being incorporated in the model. <strong>Currently I have set nBatch to 64.</strong></p>

username: ssalome
cooked: <h2>After 1 epoch is completed When I run this statement evaluation(trainloader, resnet)</h2>
<p>I get the following error<br>
pred : tensor([3, 3, 3, 3, 3, 3, 3, 5, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,<br>
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,<br>
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], device=‘cuda:0’)</p>
<p>RuntimeError Traceback (most recent call last)<br>
in ()<br>
35 loss_epoch_arr.append(loss.item())<br>
36<br>
—&gt; 37 print(‘Epoch: %d/%d, Train acc : %0.2f’ % (epoch, max_epochs, evaluation(trainloader,resnet)) )<br>
38 plt.plot(loss_epoch_arr)<br>
39 plt.show()</p>
<p>1 frames<br>
/usr/local/lib/python3.6/dist-packages/torch/tensor.py in wrapped(*args, **kwargs)<br>
26 def wrapped(*args, **kwargs):<br>
27 try:<br>
—&gt; 28 return f(*args, **kwargs)<br>
29 except TypeError:<br>
30 return NotImplemented</p>
<p>RuntimeError: The size of tensor a (64) must match the size of tensor b (15) at non-singleton dimension 1</p>
<p>Actually I’m doing multilabel(multi-class) classifcation for the first time, so I dont know what changes need to be made in the evaluation function…I just know that BCEWithLogitLoss/BCELoss is used</p>

username: ssalome
cooked: <p>Can someone help me out with this…cause its urgent <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_frowning_face.png?v=9" title=":slightly_frowning_face:" class="emoji" alt=":slightly_frowning_face:"></p>

username: ptrblck
cooked: <p>It seems that your evaluation code creates a shape mismatch.<br>
For a multi-label classification the shape of the model output and target should be equal, while it seems that one tensor might be missing the batch size.<br>
Could you print the shapes of the model output and target before the error is raised (in the validation code)?</p>

username: KFrank
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="18" data-topic="81553" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>It seems that your evaluation code creates a shape mismatch.<br>
For a multi-label classification the shape of the model output and target should be equal,</p>
</blockquote>
</aside>
<p>Her problem (among other things) is that she calls <code>argmax()</code> to get<br>
a single class label (for the accuracy calculation in the evaluation<br>
code), but it’s a multi-label problem, so she needs a vector of length<br>
<code>nClass</code> binary predictions.</p>
<p>I wasn’t able to explain this clearly.</p>
<p>Best.</p>
<p>K. Frank</p>

username: ptrblck
cooked: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>!</p>
<p>Ah OK, didn’t realize it and the posted code snippets were a bit hard to follow.</p>
<p><a class="mention" href="/u/ssalome">@ssalome</a><br>
You can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>