Crash with loss.backward() and autograd

Laurick: <p>Hello,</p>
<p>I am failing to understand how to use backward or autograd <img src="https://discuss.pytorch.org/images/emoji/apple/woozy_face.png?v=12" title=":woozy_face:" class="emoji" alt=":woozy_face:" loading="lazy" width="20" height="20"></p>
<p>My software keeps crashing each time when performing “auto gradients = torch::autograd::grad({ output }, { input }, { grad_output }, true);”</p>
<p>I also tried examples with loss.backward() only and it also crashed</p>
<pre><code class="lang-auto">// Choisir l'appareil : CPU ou GPU
	torch::Device device(torch::kCPU); // Utilisez torch::kCPU pour le CPU

	auto model = torch::nn::Linear(4, 3);

	model-&gt;to(device);

	// Génération de données d'entrée et de cible et déplacement sur l'appareil choisi
	auto input = torch::randn({ 3, 4 }).to(device);
	input.requires_grad_(true);

	auto target = torch::randn({ 3, 3 }).to(device);

	auto output = model(input);

	// Calculate loss
	auto loss = torch::nn::MSELoss()(output, target);

	// Utilisation de la norme des gradients comme pénalité
	auto grad_output = torch::ones_like(output).to(device);

	// Calcul des gradients
	auto gradients = torch::autograd::grad({ output }, { input }, { grad_output }, true);
	auto gradient = gradients[0];

	auto gradient_penalty = torch::pow((gradient.norm(2, /*dim=*/1) - 1), 2).mean();

	// Add gradient penalty to loss
	auto combined_loss = loss + gradient_penalty;
	combined_loss.backward();

</code></pre>
<p>Do you know what could cause these crashes?</p>
<p>Laurick</p>

Laurick: <p>Before crashing, the software returns the following values :</p>
<p>Input:<br>
0.6896  0.0109 -0.8927 -0.6019<br>
-1.2692 -0.3748  1.2181  1.2204<br>
1.5659  0.6650  0.2665  0.3639<br>
[ CUDAFloatType{3,4} ]</p>
<p>Target:<br>
1.1027  1.2941  0.5516<br>
-1.4952 -0.2794  0.0011<br>
-0.2567  1.1541  0.6320<br>
[ CUDAFloatType{3,3} ]</p>
<p>Output:<br>
0.1708  0.4900 -0.3538<br>
-0.1719  0.5131 -0.8069<br>
0.8201 -0.4959 -0.9887<br>
[ CUDAFloatType{3,3} ]</p>
<p>grad_output:<br>
1  1  1<br>
1  1  1<br>
1  1  1<br>
[ CUDAFloatType{3,3} ]</p>

ptrblck: <p>Could you describe what exactly is crashing and post the error message here, please?</p>

Laurick: <p>Hello,</p>
<p>Thank you for your answer</p>
<p>Sure, this is the message I get :</p>
<p>“exception Microsoft C++ : c10::Error à l’emplacement de mémoire 0x0000009C920FD390”</p>

ptrblck: <p>This seems to point to a memory violation on the host. Could you try to create the stacktrace?</p>

Laurick: <aside class="quote no-group" data-username="ptrblck" data-post="5" data-topic="205504">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>stacktrace</p>
</blockquote>
</aside>
<p>The versions I use are :<br>
C++17<br>
Visual Studio 2019<br>
Libtorch : 2.0.0+cu118</p>
<p>The stacktrace recognizes everything in the code expect when calling for external code:</p>
<pre><code>c10.dll!00007ffe8ce9ce9e()	Inconnu
torch_cpu.dll!00007ffe5f2b1e16()	Inconnu
torch_cpu.dll!00007ffe5f2b19fa()	Inconnu
torch_cpu.dll!00007ffe5f2eb48a()	Inconnu
torch_cpu.dll!00007ffe5cad7f34()	Inconnu
torch_cpu.dll!00007ffe5c8f8b3f()	Inconnu
</code></pre>
<p>It seems that the software can’t read c10.dll and torch_cpu.dll ; I don’t know why torch_cuda.dll is not called here</p>
<p>This issue seems to be related to <a href="https://github.com/pytorch/pytorch/issues/73912" class="inline-onebox" rel="noopener nofollow ugc">libtorch model predict cuda convert to cpu: C10::error at memory location · Issue #73912 · pytorch/pytorch · GitHub</a></p>