torch::nn::CrossEntropyLoss does no work

sereal96: <p>Hi , I was traying to translate this example:<br>
<a href="https://github.com/python-engineer/pytorchTutorial/blob/master/11_softmax_and_crossentropy.py" class="inline-onebox" rel="noopener nofollow ugc">pytorchTutorial/11_softmax_and_crossentropy.py at master · python-engineer/pytorchTutorial · GitHub</a> , lines 57-70<br>
from “Python-PyTorch” to “C++ - PyTorch”</p>
<h2>TEST01</h2>
<pre><code class="lang-auto"># target is of size nSamples = 1
# each element has class label: 0, 1, or 2
# Y (=target) contains class labels, not one-hot
Y = torch.tensor([0])

# input is of size nSamples x nClasses = 1 x 3
# y_pred (=input) must be raw, unnormalizes scores (logits) for each class, not softmax
Y_pred_good = torch.tensor([[2.0, 1.0, 0.1]])
Y_pred_bad = torch.tensor([[0.5, 2.0, 0.3]])
l1 = loss(Y_pred_good, Y)
l2 = loss(Y_pred_bad, Y)

print(f'PyTorch Loss1: {l1.item():.4f}')
print(f'PyTorch Loss2: {l2.item():.4f}')
</code></pre>
<p>My translation C++</p>
<pre><code class="lang-auto">torch::nn::CrossEntropyLoss criterion;

	auto Y = torch::tensor({0}, torch::kFloat32);

	auto Y_good = torch::tensor({ {2.0, 1.0, 0.1} }, torch::kFloat32);
	auto Y_bad = torch::tensor({ {0.5, 2.0, 0.3} }, torch::kFloat32);

	auto l1 = criterion(Y_good, Y);//&lt;-------------ERROR????

	std::cout &lt;&lt; l1 &lt;&lt; std::endl;
</code></pre>
<p>But every time, it gave me an error</p>
<p>Unhandled exception at 0x00007FF80AE7D759 in Test01.exe: Microsoft C++ exception: c10::Error at memory location 0x000000E17614A6F0.</p>
<p>It is possible that there is an error, because I didn’t use any “forward” method.  So I create a SECOND test with a linear regression that use “forward” to test this possibility.  (I base my second try using this tutorial <strong>https:/ / github . com/prabhuomkar/pytorch-cpp/blob/master/tutorials/popular/blitz/training_a_classifier/src/main.cpp</strong>), because making things harder always solve the problem <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>
<h2>TEST02</h2>
<p>class “MyNetImpl”  ---- mynnet.h</p>
<pre><code class="lang-auto">#pragma once
#include &lt;torch/torch.h&gt;
class MyNetImpl : public torch::nn::Module 
{
public:
    MyNetImpl();
    torch::Tensor forward(torch::Tensor x);
private:
    torch::nn::Linear line1;    
};
TORCH_MODULE(MyNet);
</code></pre>
<p>with mynnet.cpp</p>
<pre><code class="lang-auto">#include "mynnet.h"

MyNetImpl::MyNetImpl():
	line1(torch::nn::LinearOptions(9, 2))
{
	register_module("line1", line1);
}

torch::Tensor MyNetImpl::forward(torch::Tensor x)
{
	auto out = line1-&gt;forward(x);
	return out;	
}
</code></pre>
<p>in the main, I use the dataset from (<a href="https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/" class="inline-onebox" rel="noopener nofollow ugc">Index of /ml/machine-learning-databases/breast-cancer-wisconsin</a>), it has:</p>
<p>Attribute          ||-----           Domain</p>
<hr>
<ol>
<li>Sample code number             ||----- id number</li>
<li>Clump Thickness                    ||----- 1 - 10</li>
<li>Uniformity of Cell Size       ||-----1 - 10</li>
<li>Uniformity of Cell Shape      ||-----1 - 10</li>
<li>Marginal Adhesion             ||-----1 - 10</li>
<li>Single Epithelial Cell Size   ||-----1 - 10</li>
<li>Bare Nuclei                   ||-----1 - 10</li>
<li>Bland Chromatin               ||-----1 - 10</li>
<li>Normal Nucleoli               ||-----1 - 10</li>
<li>Mitoses                       ||-----1 - 10</li>
<li>Class:                        ||-----(2 for benign, 4 for malignant)</li>
</ol>
<p>Important:<br>
Using torch::nn::BCELoss works well, however y want to use torch::nn::CrossEntropyLoss, for that reason the <strong>11.Class</strong> is loaded like these:</p>
<p>benign<br>
{0,1}</p>
<p>malignant<br>
{1,0}</p>
<p>with that I can use: <em><strong>torch::nn::LinearOptions(9, 2)</strong></em></p>
<p>the method LoadereSCV return a pair of tensors, that are in this format:</p>
<pre><code class="lang-auto">auto targets = torch::empty({ size_, 2 }, torch::kFloat32);
auto input = torch::empty({ size_, 9 }, torch::kFloat32);
</code></pre>
<pre><code class="lang-auto">void Test02()
{
	std::string path = "//breast-cancer-wisconsin.data";

	int dataloaderNum = 699;
	int num_epochs = 100;
	float learning_rate = 0.01;
	torch::DeviceType device_type;

	if (torch::cuda::is_available()) {
		std::cout &lt;&lt; "CUDA available! Training on GPU." &lt;&lt; std::endl;
		device_type = torch::kCUDA;
	}
	else {
		std::cout &lt;&lt; "Training on CPU." &lt;&lt; std::endl;
		device_type = torch::kCPU;
	}


	auto train_dataset = LoadereSCV(path, 0)
		.map(torch::data::transforms::Stack&lt;&gt;());
	auto train_loader = torch::data::make_data_loader&lt;torch::data::samplers::RandomSampler&gt;(
		std::move(train_dataset), dataloaderNum);


	MyNet net = MyNet();

	net-&gt;to(device_type);

	// // Define a Loss function and optimizer
	torch::nn::CrossEntropyLoss criterion;// (torch::nn::CrossEntropyLossOptions().ignore_index(-100).reduction(torch::kMean));

	torch::optim::SGD optimizer(net-&gt;parameters(), torch::optim::SGDOptions(0.001).momentum(0.9));
	//torch::optim::Adam optimizer(net-&gt;parameters(), torch::optim::AdamOptions());


	// Train the network
	for (size_t epoch = 0; epoch &lt; num_epochs; ++epoch)
	{
		double running_loss = 0.0;

		int i = 0;
		for (auto&amp; batch : *train_loader)
		{
			// get the inputs; data is a list of [inputs, labels]
			auto inputs = batch.data.to(device_type);
			auto labels = batch.target.to(device_type);

			// zero the parameter gradients
			optimizer.zero_grad();

			// forward + backward + optimize
			auto outputs = net-&gt;forward(inputs);
			std::cout &lt;&lt; "inputs" &lt;&lt; std::endl;
			std::cout &lt;&lt; inputs &lt;&lt; std::endl;
			std::cout &lt;&lt; "labels" &lt;&lt; std::endl;
			std::cout &lt;&lt; labels &lt;&lt; std::endl;
			std::cout &lt;&lt; "outputs" &lt;&lt; std::endl;
			std::cout &lt;&lt; outputs &lt;&lt; std::endl;
			
			
            auto loss = criterion(outputs, labels);//&lt;---ERROR-------

			loss.backward();
			optimizer.step();

			// print statistics
			running_loss += loss.item&lt;double&gt;();
			if (i % 10 == 0)
			{
				std::cout &lt;&lt; "[" &lt;&lt; epoch + 1 &lt;&lt; ", " &lt;&lt; i &lt;&lt; "] loss: "
					&lt;&lt; running_loss &lt;&lt; '\n';
				running_loss = 0.0;
			}
			i++;
		}
		std::cout &lt;&lt; "Epoch = " &lt;&lt; epoch &lt;&lt; " i=" &lt;&lt; i - 1 &lt;&lt; " \n" &lt;&lt; std::endl;
	}
	std::cout &lt;&lt; "Finished Training\n\n";
}

int main()
{
Test02();
system("pause");
return 0;
}
</code></pre>
<p>These are my:</p>
<pre><code class="lang-auto">inputs
 0.5000  0.6000  0.6000  0.8000  0.6000  1.0000  0.4000  1.0000  0.4000
 0.3000  0.1000  0.4000  0.1000  0.2000  0.1000  0.1000  0.1000  0.1000
 0.7000  0.5000  0.3000  0.7000  0.4000  1.0000  0.7000  0.5000  0.5000
 0.1000  0.1000  0.1000  0.1000  0.2000  0.2000  0.1000  0.1000  0.1000
 0.5000  0.2000  0.2000  0.2000  0.1000  0.1000  0.2000  0.1000  0.1000
.
.
.
.
 0.1000  0.1000  0.1000  0.1000  0.1000  0.1000  0.2000  0.1000  0.1000
 1.0000  0.6000  0.6000  0.2000  0.4000  1.0000  0.9000  0.7000  0.1000
 0.2000  0.3000  0.1000  0.1000  0.5000  0.1000  0.1000  0.1000  0.1000
 0.6000  1.0000  1.0000  1.0000  0.8000  1.0000  1.0000  1.0000  0.7000
 0.8000  1.0000  1.0000  1.0000  0.8000  1.0000  1.0000  0.7000  0.3000
[ CUDAFloatType{699,9} ]
</code></pre>
<pre><code class="lang-auto">labels
 0  1
 1  0
 0  1
 0  1
 0  1
.
.
.
 0  1
 0  1
 0  1
 0  1
 1  0
[ CUDAFloatType{699,2} ]
</code></pre>
<pre><code class="lang-auto">outputs
-0.0505  0.2318
 0.0214  0.1086
-0.0315  0.2268
-0.0436  0.1762
 0.0056  0.2524
.
.
.
-0.0477  0.2313
-0.0516  0.2239
-0.0434  0.2346
 0.0050  0.2397
 0.2576  0.1424
[ CUDAFloatType{699,2} ]
</code></pre>
<p>Well that’s it,  could somebody help me with this.  In these 2 tests torch::nn::CrossEntropyLoss does not work, however if you use<br>
github . com  /prabhuomkar / pytorch-cpp/blob/master/tutorials/popular/blitz/training_a_classifier/src/main.cpp<br>
tutorial.  It works fine.  Even with my custom dataset of image 36x36.</p>

sereal96: <p>I find the ERROR in test02,  the problem appears when I created the labels.  I send</p>
<pre><code class="lang-auto">targets.to(torch::kFloat32)
</code></pre>
<p>but it should be:</p>
<pre><code class="lang-auto">targets.to(torch::kInt64)
</code></pre>
<p>after that change everything works great, for test02</p>