About the running errors with CUDA in C++

Chen0729: <p>Hi, All</p>
<p>I have a running error with CUDA in C++ libtorch as follows:<br>
terminate called after throwing an instance of ‘c10::Error’<br>
what():  Tensor for argument <span class="hashtag">#3</span> ‘mat2’ is on CPU, but expected it to be on GPU (while checking arguments for addmm)<br>
Exception raised from checkSameGPU at /pytorch/aten/src/ATen/TensorUtils.cpp:122 (most recent call first):<br>
It seems to me that it is a conflict between GPU and CPU. But I am not sure exactly what is the conflict and how to correct it.<br>
The related code is:<br>
torch::Tensor forward(int64_t batch_size, bool cuda = false){<br>
torch::Tensor x=torch::autograd::Variable(torch::rand({batch_size,z_dim}));<br>
if(cuda)<br>
x = x.cuda();<br>
x=torch::nn::functional::softplus(bn1(fc1(x))+bn1_b);</p>
<p>and before that I have a line related to CPU:<br>
I have a line like:<br>
return loss.data().cpu();</p>
<p>Shall I modify this line to other ways ? Any comments please ?</p>
<p>Thanks</p>

glaringlee: <p><a class="mention" href="/u/chen0729">@Chen0729</a><br>
This might not be conflict. This is caused by mixed CPU tensor (storage is in main memory) and GPU tensor (storage is on GPU) together in a function.<br>
Check your code, I guess there should be somewhere that you passed both CPU and GPU tensors together to a function.</p>

Chen0729: <p>Thanks so the line that triggers the error is at:<br>
“x=torch::nn::functional::softplus(bn1(fc1(x))+bn1_b);”<br>
x is defined as<br>
torch::Tensor x=torch::autograd::Variable(torch::rand({batch_size,z_dim}));</p>
<p>The code does not allow me to use</p>
<pre><code>    "assert(x.device().type()==torch::kCUDA);"
</code></pre>
<p>So in the debugger, how shall I check if x is CPU tensor or GPU tensor please ? Are there any other tensors that I need to check ? Thanks.</p>
<p>bn1, fc1 and bn1_b are defined as:</p>
<p>GeneratorImpl(int64_t z_dim, int64_t output_dim){</p>
<pre><code>    fc1 = register_module("fc1", torch::nn::Linear(torch::nn::LinearOptions(z_dim, 500).bias(false)));

    bn1 = register_module("bn1", torch::nn::BatchNorm1d(torch::nn::BatchNorm1dOptions(500).eps(1e-6).momentum(0.5).affine(false)));
  
    fc2 = register_module("fc2", torch::nn::Linear(torch::nn::LinearOptions(500, 500).bias(false)));
  
    bn2 = register_module("bn2", torch::nn::BatchNorm1d(torch::nn::BatchNorm1dOptions(500).eps(1e-6).momentum(0.5).affine(false)));
 
    fc3 = register_module("fc3",LinearWeightNorm(500, output_dim, 1));
    bn1_b = register_parameter("bn1_b",torch::zeros(500));
    bn2_b = register_parameter("bn2_b",torch::zeros(500));
    torch::nn::init::xavier_uniform_(fc1-&gt;weight);
    torch::nn::init::xavier_uniform_(fc2-&gt;weight);
};</code></pre>

glaringlee: <p><a class="mention" href="/u/chen0729">@Chen0729</a><br>
I guess the problem is here:<br>
bn1_b = register_parameter(“bn1_b”,torch::zeros(500));<br>
bn2_b = register_parameter(“bn2_b”,torch::zeros(500));<br>
Both zero tensor are CPU tensors.</p>
<p>x was converted to gpu but bn1_b is not.<br>
x=torch::nn::functional::softplus(bn1(fc1(x))+bn1_b)</p>
<p>And I am wondering what is the error message when you do assert? assert should be doable.</p>

Chen0729: <p>Thanks for your comment.</p>
<p>Really appreciate that.</p>
<p>Best,</p>