How to print CrossEntropyLoss of data

username: oasjd7
cooked: <p>I want to test nn.CrossEntropyLoss() is same as tf.nn.softmax_cross_entropy_with_logits in tensorflow.<br>
so I have tested on tensorflow and pytorch. I got value with tensorflow, but I don`t know how to get value of pytorch.</p>
<p>Tensorflow test :</p>
<pre><code class="lang-auto">    sess = tf.Session()
    y_true = tf.convert_to_tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]))
    y_hat = tf.convert_to_tensor(np.array([[0.5, 1.5, 0.1], [2.2, 1.3, 1.7]]))

    # print(y_true)
    tf_loss = tf.nn.softmax_cross_entropy_with_logits(labels=y_hat, logits=y_true)
    print(sess.run(tf_loss))
   -&gt; [1.7580339  6.36751251]
</code></pre>
<p>Pytorch test :</p>
<pre><code class="lang-auto">    loss = nn.CrossEntropyLoss()
    input = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]))
    target = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]), dtype=torch.long)
    print(type(input), type(target))
    output = loss(input, target)
    output.backward()
    print(output)
It occurs error :weary:
&gt;        target = torch.Tensor(np.array([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]), dtype=torch.long)
&gt; TypeError: new() received an invalid combination of arguments - got (numpy.ndarray, dtype=torch.dtype), but expected one of:
&gt;  * (torch.device device)
&gt;  * (torch.Storage storage)
&gt;  * (Tensor other)
&gt;  * (tuple of ints size, torch.device device)
&gt;       didn't match because some of the keywords were incorrect: dtype
&gt;  * (object data, torch.device device)
&gt;       didn't match because some of the keywords were incorrect: dtype

</code></pre>
<p>Anyone can help me?</p>

username: ptrblck
cooked: <p>I’ve fixed your PyTorch code as there were some minor issues:</p>
<pre><code class="lang-python">loss = nn.CrossEntropyLoss(reduction='none')
input = torch.tensor([[0.5, 1.5, 0.1], [2.2, 1.3, 1.7]], requires_grad=True)
target = torch.tensor([1, 2], dtype=torch.long)
print(type(input), type(target))
output = loss(input, target)
output.backward(torch.ones_like(target, dtype=torch.float))
print(output)
</code></pre>
<p>The first one was that <code>nn.CrossEntropyLoss</code> does not use one-hot encoded targets, but targets storing the class index. To get the loss for both elements, we also have to pass <code>reduction='none'</code> as otherwise the default will be <code>elementwise_mean</code>.</p>
<p>If you want to cast a numpy array in to a <code>tensor</code>, you should use <code>torch.from_numpy</code> instead if directly creating it.</p>
<p>In your tensorflow code it looks like you’ve mixed up <code>labels</code> and <code>logits</code>.</p>

username: oasjd7
cooked: <p>I exchanged y_true and y_hat in tensorflow code.<br>
and I got same results on both tensorflow and pytorch codes.<br>
Thanks !!!</p>

username: blade
cooked: <p>I was trying to reproduce this test for an input with size <code>torch.Size([100, 9, 9, 4])</code> and target with size <code>torch.Size([100, 9, 9])</code>, but got error.</p>
<blockquote>
<p>RuntimeError: Assertion `input0 == target0 &amp;&amp; input2 == target1 &amp;&amp; input3 == target2’ failed.</p>
</blockquote>
<p>I was wondering what is the best way to support batch size, etc. for computing the loss. In my attempt, I used <code>input.view(-1,4)</code> with <code>target.view(-1,4).argmax()</code>. Is there a better way to do it?</p>

username: ptrblck
cooked: <p>Could you post a code snippet to reproduce this issue?</p>

username: blade
cooked: <pre><code class="lang-auto">    output = torch.rand(2,4,4,2,requires_grad=True)

    target = torch.tensor([[[[0, 1],
              [1, 0],
              [1, 0],
              [0, 1]],

             [[1, 0],
              [0, 1],
              [0, 1],
              [1, 0]],

             [[0, 1],
              [0, 1],
              [1, 0],
              [1, 0]],

             [[0, 1],
              [1, 0],
              [0, 1],
              [0, 1]]],

            [[[0, 1],
              [0, 1],
              [1, 0],
              [0, 1]],

             [[0, 1],
              [0, 1],
              [0, 1],
              [1, 0]],

             [[1, 0],
              [1, 0],
              [1, 0],
              [0, 1]],

             [[1, 0],
              [1, 0],
              [0, 1],
              [0, 1]]]])

    output = output.view(-1,4)
    target = target.view(-1,4).argmax(1)
    loss = torch.nn.CrossEntropyLoss(reduction='none')
    my_loss = loss(output, target).view(-1,4,4)
</code></pre>
<p>I didn’t know of a more graceful way to produce random label. Would have been more appealing to eye if I did.</p>

username: ptrblck
cooked: <p>Your code snippet doesn’t produce this error using <code>1.4.0.dev20191109</code>.<br>
Which PyTorch version are you using?</p>