How to apply a element wise cross_entropy/nll loss to a 3D matrix?

username: Ryanzsun
cooked: <p>Hi guys, I’m working on a 3D Unet model. Currently I’m facing a trouble in implementing the loss function for 3D tensor. The output of my model is 1 x 2 x 64 x 64 x 64 (batchsize, class, H, W , Layer) and the groudntruth is<br>
1 x 64 x 64 x 64. However,  the nll_loss function does not allow me to input a 5D tensor. It seems different from the documentation. Could someone please help me with this problem?</p>

username: richard
cooked: <p>What version of pytorch are you using?</p>
<p>You might be looking at the master version of the docs: <a href="http://pytorch.org/docs/master/nn.html?highlight=nllloss#torch.nn.NLLLoss" rel="nofollow noopener">http://pytorch.org/docs/master/nn.html?highlight=nllloss#torch.nn.NLLLoss</a>. NLLLoss for multiple dimensions was recently added; you’ll have to build from source to use it.</p>

username: Ryanzsun
cooked: <p>I see, my current version is 0.3.0 from conda. Could you please give me any suggestion to solve this problem in 0.3.0?</p>

username: richard
cooked: <p>In 0.3.0, you’d have to do the following workaround. You can reshape your input into a 4D tensor and your target into a 3D tensor, and send them into the loss function, as so:</p>
<pre><code class="lang-auto">input_4d = input.view(1, 2, 64, -1)
target_3d = target.view(1, 64, -1)
loss = nn.NLLLoss(reduce=False)
out_3d = loss(input_4d, target_3d)
out = out_3d.view(1, 64, 64, 64)
</code></pre>

username: Ryanzsun
cooked: <p>This is really helpful. Thanks a lot.</p>

username: richard
cooked: <p>The next version of pytorch will have this behavior built-in, as you’ve seen in the docs!</p>

username: sank
cooked: <p><a class="mention" href="/u/ryanzsun">@Ryanzsun</a>  can you tell me how you converted 3D images into channels, as described in the paper<br>
like X x Y x Z x 3( how a medical data with n number of slices can have 3 channels) did you converted  each slice to RGB?</p>

username: Ryanzsun
cooked: <p>Actually I didn’t convert the image into rgb channels. Most of the medical image contains only one channel. This question was about the output format of NN.</p>

username: sank
cooked: <p><a class="mention" href="/u/ryanzsun">@Ryanzsun</a> yes exactly, Actually i am trying to implement 3DUNet, but the model has (X,Y,Z,3)  dimensions as input, and in the paper it is written that there are 3 channels, so what are these 3 channels then?</p>

username: Ryanzsun
cooked: <p>My data has only one dimension which is HU value for MRI image. In some cases, for example color ultrasonography, may contains rgb channels. It’s totally depends on your dataset. You should change your model according to the data, you don’t have to stick to the paper.</p>

username: sajastu
cooked: <p>What about pytorch 1.1.0 for cross entropy loss? My input is 3d: [<span class="hashtag">#batch_size</span>, <span class="hashtag">#class_index</span>, <span class="hashtag">#scores</span>] , while the target is 2d: [<span class="hashtag">#batch_size</span>, <span class="hashtag">#class_index</span>]. I’ve asked the question here, but couldn’t find a good solution yet: <a href="https://discuss.pytorch.org/t/cross-entropy-loss-for-sentence-classification/74486/3" class="inline-onebox">Cross entropy loss for sentence classification</a></p>

username: charan_Vjy
cooked: <p>The answer has been provided here.</p><aside class="quote quote-modified" data-post="1" data-topic="74486">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sajastu/48/14519_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/cross-entropy-loss-for-sentence-classification/74486">Cross entropy loss for sentence classification</a> 
  </div>
  <blockquote>
    I have a tensor in shape of [ #batch_size, #n_sentences, #scores]. #scores are calculated for each fixed class. To clarify, suppose we have batch size of 1, with 31 sentences and 5 classes that sentences have been assigned to. So the tensor would have the shape of [1, 31, 5]. Now as my target (i.e., true section labels of each 31 sentences), I’d have a tensor in shape of [1, 31]. containing 31 numbers in the range of 0-4. I’m wondering how I can compute cross entropy loss for these (inputs and t…
  </blockquote>
</aside>