`data_ptr` after `to` corrupts first 128 bits of CUDA tensor?

Amit_Gupta: <p>Consider the C++ example below</p>
<pre data-code-wrap="C"><code class="lang-nohighlight">#include &lt;torch/script.h&gt;
#include &lt;iostream&gt;

int main(){
    double arr1[] =  {1., 2., 3., 4., 5. , 6., 1., 2., 3., 4., 5. , 6.};
    float arr2[] =  {1., 2., 3., 4., 5. , 6., 1., 2., 3., 4., 5. , 6.};

    auto torch_double_options = torch::TensorOptions().dtype(torch::kFloat64);
    auto torch_single_options = torch::TensorOptions().dtype(torch::kFloat32);

    auto tensor1 = torch::from_blob(arr1, 12, torch_double_options).to(torch::kCUDA);
    auto tensor2 = torch::from_blob(arr2, 12, torch_single_options).to(torch::kCUDA);

    auto tensor1_ptr = tensor1.to(torch::kCPU).data_ptr&lt;double&gt;();
    auto tensor2_ptr = tensor2.to(torch::kCPU).data_ptr&lt;float&gt;();
    std::cout &lt;&lt; "\n";
    for (int i = 0; i &lt; 12; i++){
        std::cout &lt;&lt; *(tensor1_ptr + i) &lt;&lt;  "  " &lt;&lt; *(tensor2_ptr + i) &lt;&lt; "\n";
    }

    auto tensor1_cuda = tensor1.to(torch::kCUDA);
    auto tensor2_cuda = tensor2.to(torch::kCUDA);

    // auto tensor1_cpu = tensor1.to(torch::kCPU);
    // auto tensor1_ptr = tensor1_cpu.data_ptr&lt;double&gt;();
    // auto tensor2_cpu = tensor2.to(torch::kCPU);
    // auto tensor2_ptr = tensor2_cpu.data_ptr&lt;float&gt;();
    // std::cout &lt;&lt; "\n";
    // for (int i = 0; i &lt; 12; i++){
    //     std::cout &lt;&lt; *(tensor1_ptr + i) &lt;&lt;  "  " &lt;&lt; *(tensor2_ptr + i) &lt;&lt; "\n";
    // }

}
</code></pre>
<p>Generates the following output</p>
<pre><code class="lang-auto">4.99056e-315  0.0846298
1.28505e-316  0
3  5.17494e-38
4  0
5  5
6  6
1  1
2  2
3  3
4  4
5  5
6  6
</code></pre>
<p>As you can see it gives garbage values for first two digits of double tensor and 4 digits of float tensor. Hence when <code>data_ptr</code> is accessed from a tensor residing on GPU in single line <code>to().data_ptr&lt;&gt;()</code> fashion, it gives mangled results for first 128 bits?</p>
<p>Interestingly, if I split the <code>to</code> and <code>data_ptr</code> in two separate lines, the problem disappears (uncomment the commented out line).<br>
ie.</p>
<pre data-code-wrap="Cpp"><code class="lang-nohighlight">
    auto tensor1_cpu = tensor1.to(torch::kCPU);
    auto tensor1_ptr = tensor1_cpu.data_ptr&lt;double&gt;();
    auto tensor2_cpu = tensor2.to(torch::kCPU);
    auto tensor2_ptr = tensor2_cpu.data_ptr&lt;float&gt;();
    std::cout &lt;&lt; "\n";
    for (int i = 0; i &lt; 12; i++){
        std::cout &lt;&lt; *(tensor1_ptr + i) &lt;&lt;  "  " &lt;&lt; *(tensor2_ptr + i) &lt;&lt; "\n";
    }
</code></pre>
<p>Gives the proper tensor back.</p>
<pre><code class="lang-auto">1  1
2  2
3  3
4  4
5  5
6  6
1  1
2  2
3  3
4  4
5  5
6  6
</code></pre>
<p>Also it does not change the original tensor it seems, only pointer is misplaced?<br>
What am I doing wrong?</p>

Amit_Gupta: <p>May be related: <a href="https://discuss.pytorch.org/t/can-t-obtain-right-data-from-tensor-contiguous-data-ptr/156860" class="inline-onebox">Can`t obtain right data from tensor.contiguous().data_ptr()</a></p>

ptrblck: <aside class="quote no-group" data-username="Amit_Gupta" data-post="1" data-topic="166573">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amit_gupta/48/43905_2.png" class="avatar"> Amit_Gupta:</div>
<blockquote>
<p>Interestingly, if I split the <code>to</code> and <code>data_ptr</code> in two separate lines, the problem disappears (uncomment the commented out line).</p>
</blockquote>
</aside>
<p>I wanted to suggest the same approach, as I believe</p>
<pre><code class="lang-auto">auto tensor1_ptr = tensor1.to(torch::kCPU).data_ptr&lt;double&gt;();
</code></pre>
<p>would create a temp. tensor on the CPU and directly lose the reference it it afterwards.<br>
Reading its values might thus be UB and I would make sure the actual tensor you are reading from is alive and assigned to a valid reference.</p>