CUDA 11.6 extension problem

Jakub_Mitura: <p>Hello I am trying to use my CUDA kernel in Pytorch python. PyTorch 1.10.2+cu113 on Visual studio 2019 with CUDA 11.6 installed , pybind 2.8.1 ; Python 3,7</p>
<p>In documentation of pytorch  and its forums it is indicated tha one shoud use Aten library instead of pytorch.extension in order to avoid</p>
<pre><code class="lang-auto">"too few arguments for template template parameter "Tuple"  
</code></pre>
<p>And those workaround works Hovewer When I try to include the pybind into my .cu file by</p>
<pre><code class="lang-auto">#include &lt;pybind11/pybind11.h&gt;
</code></pre>
<p>I get the error mentioned above</p>
<p>If I will comment out</p>
<pre><code class="lang-auto">//#include &lt;pybind11/pybind11.h&gt;
</code></pre>
<p>all compiles - and .pyd files is produced into a solution folder - Although until I do not have connection to python file I do not know wheather it works</p>
<p>Detailed description of the configuration and minimal example Code I use</p>
<ol>
<li>Installation of Python 3.7 with debug Libraries and download from Pytorch libtorch-win-shared-with-deps-debug-1.10.2+cu113</li>
<li>in vcpkg - install Version: 2.8.1; triple set to Windows 64</li>
</ol>
<pre><code class="lang-auto">.\vcpkg install pybind11 
.\vcpkg integrate install
</code></pre>
<ol start="2">
<li>In Visual studio 2019 - new CUDA 11.6 project</li>
<li>Set architecture to x64</li>
<li>Editing properties - if not stated otherwise editing for all configrations<br>
a) Vc++ Directories - to include directories</li>
</ol>
<ul>
<li>add Python37/include folder</li>
<li>add …\libtorch-win-shared-with-deps-debug-1.10.2+cu113\libtorch\include</li>
<li>add …\libtorch-win-shared-with-deps-debug-1.10.2+cu113\libtorch\torch\csrc\api\include<br>
b) Vc++ Directories - to Library directories</li>
<li>add Python37/lib folder<br>
c) Linker - Additional Dependencies - for debug python37_d.lib for release python37.lib<br>
d) C/C++ → Language → Conformance mode to No<br>
e) General tab- Additional Library Directories-  …\libtorch-win-shared-with-deps-debug-1.10.2+cu113\libtorch\lib<br>
f) Input Tab - additional dependencies - adding torch.lib, torch_cuda.lib, caffe2_nvrtc.lib, c10.lib, c10_cuda.lib, torch_cpu.lib<br>
g) Build Events → Post-Build Events →xcopy $(ProjectDir)…\libtorch-win-shared-with-deps-debug-1.10.2+cu113\libtorch\lib*.dll $(SolutionDir)$(Platform)$(Configuration)\ /c /y<br>
h) General - Advanced - Target File Extension - changed to .pyd<br>
i) CUDA C/C++  - (-rdc=true)<br>
j) CUDA C/C++ - Device - code generation - compute_75,sm_75<br>
k) CUDA Linker - General - Additional Library directories added libtorch\lib</li>
</ul>
<p>Currently I have single file ( did not got to Python file Yet) lltm_cuda_kernel.cu inside I have simple code</p>
<pre><code class="lang-auto">
#include "cuda_runtime.h"
#include "cuda.h"
#include "device_launch_parameters.h"
#include "stdio.h"


#include &lt;ATen/ATen.h&gt;
#include &lt;Python.h&gt;
#include &lt;ATen/Parallel.h&gt;
#include &lt;ATen/Dispatch.h&gt;
#include &lt;ATen/core/op_registration/op_registration.h&gt;
#include &lt;ATen/cuda/CUDAContext.h&gt;
#include &lt;ATen/native/TensorIterator.h&gt;
#include &lt;ATen/quantized/Quantizer.h&gt;
#include &lt;c10/cuda/CUDAGuard.h&gt;
#include &lt;torch/library.h&gt;
#include &lt;ATen/native/StridedRandomAccessor.h&gt;
#include &lt;cooperative_groups.h&gt;
#include &lt;cooperative_groups/reduce.h&gt;

#include &lt;pybind11/pybind11.h&gt;

#pragma once		
        template &lt;typename scalar_t&gt;
        __global__ void lltm_cuda_forward_kernel(        
             at::PackedTensorAccessor&lt;scalar_t, 1, at::RestrictPtrTraits&gt; input,
             at::PackedTensorAccessor&lt;scalar_t, 1, at::RestrictPtrTraits&gt; output,int  length){

            for (int i = blockIdx.x * blockDim.x + threadIdx.x; i &lt; length; i += blockDim.x * gridDim.x) {
                output[i] = input[i] + 1;
            }
        } 


#pragma once
static void testAdd(
    at::Tensor inputtt,
    at::Tensor output) {
    int blocks = 10;
    int threads = 512;

    cudaStream_t stream = at::cuda::getCurrentCUDAStream();
    int length = 5;

        switch (inputtt.type().scalarType()) {
    case at::ScalarType::Float:
        lltm_cuda_forward_kernel&lt;float&gt; &lt;&lt; &lt;blocks, threads, 0, stream &gt;&gt; &gt; (inputtt.packed_accessor64&lt;float, 1, at::RestrictPtrTraits&gt;()
            , output.packed_accessor64&lt;float, 1, at::RestrictPtrTraits&gt;(), length);
    
    
    }


}

int main() {

    printf("  *** ");

    return 0;
}
</code></pre>
<p>What am I doing wrong - is there a better way ? For some obscure  reason I can achieve greater occupancy with my kernel using Visual studio project than in the cmake - but I can compile it also with cmake if it needs to be so.</p>
<p>I was also trying to create a separate cpp file and there invoke kernel by including .cu file but cooperative group libraries gave a lot of compile  errors  in such situation.</p>
<p>sources</p>
<p>            <iframe src="https://www.youtube.com/embed/-eIkUnCLMFc?feature=oembed&amp;wmode=opaque&amp;list=PLb9uFnQyeGTcKIHNUNUUuLbRhumAZd-fy" width="480" height="360" frameborder="0" allowfullscreen="" seamless="seamless" sandbox="allow-same-origin allow-scripts allow-forms allow-popups allow-popups-to-escape-sandbox allow-presentation"></iframe></p><aside class="onebox allowlistedgeneric" data-onebox-src="https://towardsdatascience.com/setting-up-a-c-project-in-visual-studio-2019-with-libtorch-1-6-ad8a0e49e82c">
  <header class="source">

      <a href="https://towardsdatascience.com/setting-up-a-c-project-in-visual-studio-2019-with-libtorch-1-6-ad8a0e49e82c" target="_blank" rel="noopener nofollow ugc" title="02:03AM - 28 November 2020">Medium – 28 Nov 20</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/360;"><img src="https://miro.medium.com/max/979/0*PavmPvnszD20COXg" class="thumbnail" width="690" height="360"></div>

<h3><a href="https://towardsdatascience.com/setting-up-a-c-project-in-visual-studio-2019-with-libtorch-1-6-ad8a0e49e82c" target="_blank" rel="noopener nofollow ugc">Setting up a C++ project in Visual Studio 2019 with LibTorch 1.6</a></h3>

  <p>In this tutorial I explain how to set up a LibTorch 1.6 project in Visual Studio 2019.</p>

  <p>
    <span class="label1">Reading time: 4 min read</span>
  </p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://docs.microsoft.com/en-us/visualstudio/python/working-with-c-cpp-python-in-visual-studio?view=vs-2022">
  <header class="source">

      <a href="https://docs.microsoft.com/en-us/visualstudio/python/working-with-c-cpp-python-in-visual-studio?view=vs-2022" target="_blank" rel="noopener nofollow ugc">docs.microsoft.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://docs.microsoft.com/en-us/media/logos/logo-ms-social.png" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://docs.microsoft.com/en-us/visualstudio/python/working-with-c-cpp-python-in-visual-studio?view=vs-2022" target="_blank" rel="noopener nofollow ugc">Write C++ extensions for Python - Visual Studio (Windows)</a></h3>

  <p>This article walks you through how to create a C++ extension for Python by using Visual Studio, CPython, and PyBind11, including mixed-mode debugging.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p><a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/advanced/cpp_extension.html</a></p>

Jakub_Mitura: <p>So I did not figured out How to manage it I downgraded to CUDA 11.3 and all works</p>

Nat: <p>I had the same issue, can confirm downgrade to 11.3 solved it !</p>