Crash when using torch::from_blob with CUDA

Robin_Lobel: <p>I’m using libtorch 1.4.0 on my MSI P65 laptop (Win10, RTX 2060, NV driver 441.22, Cuda 26.21.14.4122).<br>
When using the CPU, this code works as expected:</p>
<pre><code>auto tensor1=torch::zeros({1, 1, 2, 2},torch::kCPU);
tensor1[0][0][0][0]=42;
cout &lt;&lt; "tensor1:" &lt;&lt; endl &lt;&lt; tensor1 &lt;&lt; endl &lt;&lt; endl;

float tensor2Data[]{42,0,0,0};
auto tensor2=torch::from_blob(tensor2Data,{1, 1, 2, 2},torch::kCPU);
cout &lt;&lt; "tensor2:" &lt;&lt; endl &lt;&lt; tensor2 &lt;&lt; endl &lt;&lt; endl;
</code></pre>
<p>And it shows:</p>
<blockquote>
<p>tensor1:<br>
(1,1,.,.) =<br>
42   0<br>
0   0<br>
[ CPUFloatType{1,1,2,2} ]</p>
<p>tensor2:<br>
(1,1,.,.) =<br>
42   0<br>
0   0<br>
[ CPUFloatType{1,1,2,2} ]</p>
<p>Press  to close this window…</p>
</blockquote>
<p>However when switching to CUDA, with this code:</p>
<pre><code>auto tensor1=torch::zeros({1, 1, 2, 2},torch::kCUDA);
tensor1[0][0][0][0]=42;
cout &lt;&lt; "tensor1:" &lt;&lt; endl &lt;&lt; tensor1 &lt;&lt; endl &lt;&lt; endl;

float tensor2Data[]{42,0,0,0};
auto tensor2=torch::from_blob(tensor2Data,{1, 1, 2, 2},torch::kCUDA);
cout &lt;&lt; "tensor2:" &lt;&lt; endl &lt;&lt; tensor2 &lt;&lt; endl &lt;&lt; endl;
</code></pre>
<p>It crash at from_blob without any error message:</p>
<blockquote>
<p>tensor1:<br>
(1,1,.,.) =<br>
42   0<br>
0   0<br>
[ CUDAFloatType{1,1,2,2} ]</p>
<p>Press  to close this window…</p>
</blockquote>
<p>Running a model on CUDA works - it’s only by creating a CUDA tensor with from_blob that I get this crash, without any hint or explanation.<br>
Any idea what’s going wrong ? Is there a workaround I could use to easily transfer blocks of data from the RAM to a CUDA tensor ?</p>
<p>Also, I tried creating the tensors CPU-side (first code) and then do a tensor.to(torch::kCUDA) but it didn’t transfer the tensors to CUDA.<br>
I could also transfer values one by one to a CUDA tensor using <span class="chcklst-box fa fa-square-o fa-fw"></span><span class="chcklst-box fa fa-square-o fa-fw"></span><span class="chcklst-box fa fa-square-o fa-fw"></span><span class="chcklst-box fa fa-square-o fa-fw"></span> like with tensor1, but it takes a <strong>really</strong> long time.</p>

tom: <p>That’s because the data is on the CPU and you’re instructing PyTorch to treat the CPU-pointer tensor2Data as a GPU pointer. You’d need to make it a tensor first and then move to CUDA.</p>
<p>As an aside, tensor indexing as in <code>tensor1[0][0][0][0]=42;</code> is <em>not</em> efficient if you do it at scale. This statement will, under the hood, call <code>tensor1[...].fill_</code>, it is not writing to some memory location.</p>
<p>Best regards</p>
<p>Thomas</p>

Robin_Lobel: <aside class="quote no-group" data-username="tom" data-post="2" data-topic="71298">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>You’d need to make it a tensor first and then move to CUDA.</p>
</blockquote>
</aside>
<p>Are you suggesting I do this ?</p>
<pre><code>float tensor2Data[]{42,0,0,0};
auto tensor2=torch::from_blob(tensor2Data,{1, 1, 2, 2},torch::kCPU);
tensor2.to(torch::kCUDA);
</code></pre>
<p>Because I tried, but then when I do</p>
<pre><code>cout &lt;&lt; "tensor2:" &lt;&lt; endl &lt;&lt; tensor2
</code></pre>
<p>All I get is:</p>
<blockquote>
<p>tensor2:<br>
(1,1,.,.) =<br>
42 0<br>
0 0<br>
[ CPUFloatType{1,1,2,2} ]</p>
</blockquote>
<p>Which indicates tensor2 is still on the CPU (CPUFloatType), it didn’t move to the CPU.</p>

Robin_Lobel: <p>Ohhh I just figured out I actually needs to do:</p>
<pre><code>tensor2=tensor2.to(torch::kCUDA);
</code></pre>
<p>It works now <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>