Kernel Launch Deprecated: packed_accessor arguments and `Tensor.type()` alternative

eduardo4jesus: <p>When compiling the code below, there are a lot of deprecated warnings.</p>
<pre><code class="lang-auto">AT_DISPATCH_FLOATING_TYPES(x.type(), "comp_cuda",
    ([&amp;] {
        comp_cuda_kernel&lt;scalar_t&gt;&lt;&lt;&lt;grid_size, threads&gt;&gt;&gt;(
        x.data&lt;scalar_t&gt;(), h.data&lt;scalar_t&gt;(), out.data&lt;scalar_t&gt;(),
        B, F, C, H, W);
    }));
</code></pre>
<p>As far I noticed, the warnings basically are:</p>
<blockquote>
<p><strong>warning</strong>: ‘at::DeprecatedTypeProperties&amp; at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]</p>
</blockquote>
<p>And</p>
<blockquote>
<p><strong>warning</strong>: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]</p>
</blockquote>
<p>The documentation, <a href="https://github.com/pytorch/extension-cpp/blob/1031028f3b048fdea84372f3b81498db53d64d98/cuda/lltm_cuda_kernel.cu#L119" rel="noopener nofollow ugc">here</a> <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="noopener nofollow ugc">here</a>, does not provide an alternative to <code>x.type()</code>. However it does suggests a different way to refer to <code>x.data&lt;scalar_t&gt;()</code> as in</p>
<pre><code class="lang-auto">gates.packed_accessor&lt;scalar_t,3,torch::RestrictPtrTraits,size_t&gt;(),
</code></pre>
<p>What specifically are these arguments?</p>
<ul>
<li>3</li>
<li>torch::RestrictPtrTraits</li>
</ul>
<p>Would the first and last argument always be <code>scalar_t</code> and <code>size_t</code>?</p>

tom: <aside class="quote no-group" data-username="eduardo4jesus" data-post="1" data-topic="138875">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eduardo4jesus/48/20203_2.png" class="avatar"> eduardo4jesus:</div>
<blockquote>
<p>The documentation, <a href="https://github.com/pytorch/extension-cpp/blob/1031028f3b048fdea84372f3b81498db53d64d98/cuda/lltm_cuda_kernel.cu#L119" rel="noopener nofollow ugc">here</a> <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="noopener nofollow ugc">here</a>, does not provide an alternative to <code>x.type()</code>.</p>
</blockquote>
</aside>
<p>As the warning suggests <code>t.options()</code> to get something that can be used with the factory functions and the getters of individual things (e.g. <code>t.scalar_type()</code> for the ScalarType aka dtype).</p>
<p>Do you want a pointer or an array-like object? If you want the pointer you got from .data before, it’s now called .data_ptr, which is a drop-in replacement.</p>
<p>If you want an array type object: .packed_accessor32 (preferred over the version with size_t if you know it is sufficient) has 3=dimension of the tensor (fixed at compile time, checked at runtime) and the traits (DefaultPtrTraits is without extra qualification, Restricted adds the <code>__restrict__</code> to the type, meaning that the compiler may assume that the tensor does not overlap with (written-to) tensors, which allows more aggressive caching.<br>
The first is the C/CUDA data element type of the tensor (fixed at compile time, checked at runtime). The latter should ideally not be size_t but <code>int32_t</code> if that works or <code>int64_t</code> and is the datatype for the indexing arithmetic (32bit is much faster than 64bit on CUDA).</p>
<p>Best regards</p>
<p>Thomas</p>

eduardo4jesus: <p>Thank you so much for your feedback and clarification.</p>
<p>By checking <a href="https://github.com/pytorch/extension-cpp/blob/1031028f3b048fdea84372f3b81498db53d64d98/cuda/lltm_cuda_kernel.cu#L87" rel="noopener nofollow ugc">their code</a> I noticed that the variable <code>gate</code> can be accessed as a vector instead of as a pointer.</p>
<hr>
<aside class="quote no-group" data-username="tom" data-post="2" data-topic="138875">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>.packed_accessor32 (preferred over the version with size_t if you know it is sufficient)</p>
</blockquote>
</aside>
<p>With the <code>Deprecation</code> warning, it seem to be more than a matter of preference.</p>
<hr>
<aside class="quote no-group" data-username="tom" data-post="2" data-topic="138875">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>the compiler may assume that the tensor does not overlap with (written-to) tensors, which allows more aggressive caching.</p>
</blockquote>
</aside>
<p>In the case of the <code>out</code> tensor in my case, and <code>output_gate</code> from the <a href="https://github.com/pytorch/extension-cpp/blob/1031028f3b048fdea84372f3b81498db53d64d98/cuda/lltm_cuda_kernel.cu#L160" rel="noopener nofollow ugc">example</a>, the tensor provided to have data written to. This provided argument, however, it is still given using <code>packed_accessor</code>. It don’t think I understood what you mean. Could you expand a bit more on that?</p>
<hr>
<aside class="quote no-group" data-username="tom" data-post="2" data-topic="138875">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>The latter should ideally not be size_t but <code>int32_t</code> if that works or <code>int64_t</code> and is the datatype for the indexing arithmetic (32bit is much faster than 64bit on CUDA).</p>
</blockquote>
</aside>
<p>Also, in my case, the tensors are of type <code>float32</code>. However, I think you are referring to the <code>index</code> type, right? It never occurred to me to think about the indexes type. I think <code>int32_t</code> should be enough for my cases since I don’t plan having a tensor that has size bigger than <code>2^32</code>.</p>
<hr>
<p>Thank you for your time.</p>