1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [1, 3, 375, 1242]

Xiaoyu_Song: <p>My batch size is 1, my image is 375  *1242 . *3, I’ve changed the numpy image to tensor image as 3 . <em>375</em> . 1242.<br>
When I call</p>
<pre><code class="lang-auto">criterion = nn.CrossEntropyLoss().cuda()
</code></pre>
<pre><code class="lang-auto">loss = criterion(outputs, labels.long())
</code></pre>
<p>The error is showing as the tiltle:</p>
<pre><code class="lang-auto">RuntimeError: 1only batches of spatial targets supported (non-empty 3D tensors) but got targets of size: : [1, 3, 375, 1242]
</code></pre>
<p>I can’t figure out where is the problem, someone can help me? Thank you.</p>
<p>Here is the more detailed codes:</p>
<pre><code class="lang-auto"> for epoch in range(2):
    # loop over the dataset multiple times
    running_loss = 0.0
    for i, data in enumerate(dataloader):
        #get the inputs
        inputs, labels = data['image'], data['semantic']
        print(inputs.size())
        print(labels.size())
        # put to GPU
        labels = labels.cuda().float()
        #zero the parameter gradients
        optimizer.zero_grad()
        
        #forward + backward +optimize
        outputs = unet(inputs.cuda())
        outputs.cuda()
        print(outputs.size())
        print(labels.long().size())
        
        loss = criterion(outputs, labels.long())
        loss.backward()
        optimizer.step()
        
        #print statistics
        running_loss += loss.item()
        if i%20 ==19: 
            # print every 20 mini-batchs
            print('[%d, %5f] loss: %.3f'%(epoch +1, i+1, running_loss/20))
            running_loss = 0.0
print('Finish Training')
</code></pre>

Xiaoyu_Song: <p>I check the documentation of the nn.CrossEntropyLoss and I find that for the target, the dimension is N H W, so in my case, I use a RGB mask, which doesn’t work.<br>
When I change my target dataset to grayscale images., it works fine.<br>
I would like to know if I want to use a RGB image as target, which loss should I use instead?<br>
Thank you</p>

ptrblck: <p><code>nn.CrossEntropyLoss</code> is usually used for classification use cases.<br>
It seems you are trying to reconstruct the image somehow. If your targets are normalized tensors with values in <code>[0, 1]</code>, you could use <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.BCELoss" rel="nofollow noopener">nn.BCELoss</a>.</p>

Xiaoyu_Song: <p>Yes, I’m trying to do an image segmentation with the image and the image-mask. I will try your recommendation, thank you <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"></p>

ptrblck: <p>Thanks for the info.<br>
In that case I would stick to a classification criterion.<br>
Could you post a sample mask with its values?<br>
Usually your mask should have the shape <code>[batch_size, h, w]</code> and contain the class indices for each pixel.</p>

Xiaoyu_Song: <p>in fact, my sample mask have the shape<code>batch_size, Channel, h, w</code>, the same size as the sample image. so the problem is the <code>channel</code> here. because my sample mask is a RGB image. In this case, which Loss Function should I use?</p>

ptrblck: <p>I guess your mask has some kind of color code in RGB, e.g. red (<code>[255, 0, 0]</code>) means car, while blue (<code>[0, 0, 255]</code>) means building.<br>
If that’s the case, could you post a sample image or post the mapping directly, since we would need to transform the RGB color-coded mask into a class index mask.</p>

Xiaoyu_Song: <p>hi, thanks for replying.<br>
I have two types of mask image. One is in RGB, the other is not.</p>
<p>Now I encounter a new problem, do I need to normalize my image with values in [0,1]</p>
<p>because I have tried to build a Unet, and every time after training, the predicted mask is always a red image.<br>
Do you know where possibly is the problem?</p>

ptrblck: <p>Your input can and should probably be normalized to properly train the model.<br>
The mask however, should most likely not be normalized, as it contains some kind of class information.<br>
How is the other mask stored if not as RGB? Does it store the class indices directly for each pixel?</p>
<p>Could you post some values and the shape of an example mask you are currently using?</p>

Xiaoyu_Song: <p>Sure, here is one sample of my RGB mask:<br>
[[[107 142  35]<br>
[107 142  35]<br>
[107 142  35]<br>
…<br>
[ 70 130 180]<br>
[ 70 130 180]<br>
[ 70 130 180]]</p>
<p>[[107 142  35]<br>
[107 142  35]<br>
[107 142  35]<br>
…<br>
[ 70 130 180]<br>
[ 70 130 180]<br>
[ 70 130 180]]</p>
<p>[[107 142  35]<br>
[107 142  35]<br>
[107 142  35]<br>
…<br>
[ 70 130 180]<br>
[ 70 130 180]<br>
[ 70 130 180]]</p>
<p>…</p>
<p>[[128  64 128]<br>
[128  64 128]<br>
[128  64 128]<br>
…<br>
[128  64 128]<br>
[128  64 128]<br>
[128  64 128]]</p>
<p>[[128  64 128]<br>
[128  64 128]<br>
[128  64 128]<br>
…<br>
[128  64 128]<br>
[128  64 128]<br>
[128  64 128]]</p>
<p>[[128  64 128]<br>
[128  64 128]<br>
[128  64 128]<br>
…<br>
[128  64 128]<br>
[128  64 128]<br>
[128  64 128]]]</p>
<p>The shape is (375, 1242, 3)</p>

Xiaoyu_Song: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/4/407989adf962120818236842acbbc1ba6d6862c7.png" data-download-href="https://discuss.pytorch.org/uploads/default/407989adf962120818236842acbbc1ba6d6862c7" title="000000_10"><img src="https://discuss.pytorch.org/uploads/default/original/2X/4/407989adf962120818236842acbbc1ba6d6862c7.png" alt="000000_10" data-base62-sha1="9cmX0d2Mo8VWkKppjXBJKgSCMwn" width="690" height="208" data-dominant-color="5B5064"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">000000_10</span><span class="informations">1242×375 5.47 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
This is one example of my mask</p>

ptrblck: <p>Thanks for the example.<br>
Are you using these RGB values directly for your classes?<br>
If so, I would recommend to use a mapping such that each pixel contains only a class index in the range <code>[0, nb_classes-1]</code>.</p>

Xiaoyu_Song: <p>Sorry I’m new to pytorch, what do you mean by mapping?<br>
I put my code link below if you are interested.<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/xysong1201/image_segmentation" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/44894406?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/xysong1201/image_segmentation" target="_blank" rel="nofollow noopener">xysong1201/image_segmentation</a></h3>

<p>Contribute to xysong1201/image_segmentation development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

ptrblck: <p>I mean something like a key value pair between your color codes in RGB and the corresponding class index.<br>
E.g. <code>[128, 64, 128]</code> would map to class0.<br>
Have a look at <a href="https://discuss.pytorch.org/t/error-in-python-s-multiprocessing-library/31355/6?u=ptrblck">this post</a> for another example using grayscale images.<br>
Could you post the classes for each separate color in your segmentation mask?</p>

Xiaoyu_Song: <p>Thank you for your explanation, I’m using the  <a href="http://www.cvlibs.net/datasets/kitti/eval_semseg.php?benchmark=semantics2015" rel="nofollow noopener">KITTI semantic segmentation datasets</a>. which are conform with <a href="https://www.cityscapes-dataset.com/" rel="nofollow noopener">The Cityscapes Dataset</a>, it has 30 classes. But I didn’t find the key value pair between my colour codes and the corresponding class index.<br>
I have seen the post you mentioned, and I don’t know the class mapping value neither, could you help?</p>

ptrblck: <p>I’m not sure where to find the mapping. <a href="https://github.com/fvisin/dataset_loaders/blob/11f8cf6f84ffa25ecf32da070fbe2ea37b9e0f7d/dataset_loaders/images/kitti.py#L62" rel="nofollow noopener">Here</a> it seems a mapping is given for 11 classes.<br>
However, if you can’t find the right mapping, you could also just get all unique color codes and create your own mapping.</p>

Xiaoyu_Song: <p>Thanks a lot for the effort you put. I really appreciate it. At the moment I am getting CUDA error: out of memory. I will sort out memory issue and will update you.</p>

Xiaoyu_Song: <p>Sorry to bother again, I have more questions:</p>
<ol>
<li>In the unet example, if I do the image mapping, for example, if I have 10 classes for the labels, the last layer of the network is <code>log_softmax</code>, does it mean that the output of the network is the probability map of of every pixel.</li>
<li>If so, after using the NLLLoss and the Adam optimisation, the weight in the network is optimized, now I feed the network with a random training image, the output is a probability map, if I want to visualize it, I need to simply <code>imshow(output)</code> or I have to remap it back?</li>
</ol>

ptrblck: <ol>
<li>
<p>Yes, you will get the log probabilities with each channel corresponding to the class index.</p>
</li>
<li>
<p><code>plt.imshow</code> should work. I would try to transform the output using <code>torch.exp</code> to get the probabilities in the range <code>[0, 1]</code>, since the colormap might look more “natural”.</p>
</li>
</ol>

Xiaoyu_Song: <p>Hi, I’m wondering how to create the mapping, I saw that your mapping code in the <a href="https://discuss.pytorch.org/t/error-in-python-s-multiprocessing-library/31355/6?u=ptrblck">post</a>, how did you find the mapping relationship?<br>
And how to create my own mapping? Do you mean that I should check all color in my mask image and then find the html code <img src="https://discuss.pytorch.org/images/emoji/apple/open_mouth.png?v=6" title=":open_mouth:" class="emoji" alt=":open_mouth:"></p>