Can't compile simple example cuda kernel

Wesley_Neill: <p>Good afternoon all,</p>
<p>I have been looking at converting some cpp code that I’ve had good success to a cuda kernel to maybe squeeze even more speed out of it. I decided to try my hand at a simple kernel first, a slight tweak on the example found here <a href="https://developer.nvidia.com/blog/even-easier-introduction-cuda/" rel="noopener nofollow ugc">https://developer.nvidia.com/blog/even-easier-introduction-cuda/</a>.</p>
<p>However, when try to compile this, nvcc just hangs forever:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53c0f91d43b3faca4124460e816f56fb06f3bc1f.png" data-download-href="https://discuss.pytorch.org/uploads/default/53c0f91d43b3faca4124460e816f56fb06f3bc1f" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53c0f91d43b3faca4124460e816f56fb06f3bc1f.png" alt="image" data-base62-sha1="bWV4wF5T1BEm0GbkEY874gZvMx9" width="690" height="39" data-dominant-color="0D0F0C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1173×67 5.33 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I think that it’s not an issue with my code itself, but just in case this is it:</p>
<pre data-code-wrap="cpp"><code class="lang-cpp">#include &lt;torch/script.h&gt;

using namespace torch;

__global__ 
void add_kernel(int N, float x[], float y[], float z[]) {
    
    int64_t i = threadIdx.x;
    int64_t s = blockDim.x;

    for (int64_t ix = i; i &lt; N; i += s) {
        z[i] = x[i] + y[i];
    }
}

Tensor add_gpu(Tensor a, Tensor b) {

    int64_t N = a.size(0);
    size_t bytes = N * sizeof(float);

    Tensor result = torch::empty(N);

    float* d_a;
    float* d_b;    
    float* d_z;
    float* h_z = result.data_ptr&lt;float&gt;();
    cudaMallocManaged(&amp;d_a, bytes);
    cudaMallocManaged(&amp;d_b, bytes);
    cudaMallocManaged(&amp;d_z, bytes);

    cudaMemcpy(d_a, a.data_ptr&lt;float&gt;(), bytes, cudaMemcpyHostToDevice);
    cudaMemcpy(d_b, b.data_ptr&lt;float&gt;(), bytes, cudaMemcpyHostToDevice);

    int64_t block_size = 256;
    int64_t grid_size = (int64_t)ceil(N / block_size);

    add_kernel&lt;&lt;&lt;grid_size, block_size&gt;&gt;&gt;(N, d_a, d_b, d_z);
    cudaMemcpy(h_z, d_z, bytes, cudaMemcpyDeviceToHost);

    cudaFree(d_a);
    cudaFree(d_b);
    cudaFree(d_z);

    return result;
}

int main(void) {
    int N = 100000;
    torch::Tensor x = torch::arange(N);
    torch::Tensor y = x + 1;
    Tensor z = add_gpu(x, y);

}
</code></pre>
<p>Can anyone explain how to correctly compile and run this little test?</p>

ptrblck: <p>Is <code>nvcc</code> really hanging or is it expecting more input?<br>
I’m not familiar with Windows/MINGW64, but on Linux bash will create a newline if <code>\</code> is added as seen here:</p>
<pre><code class="lang-auto">$ ls
build  CMakeLists.txt  main.cpp  tmp.py

$ ls \
&gt; 
</code></pre>
<p>Note that my cursor drops to the new line and I can continue with my command, e.g.:</p>
<pre><code class="lang-auto">$ ls \
&gt; ./build
CMakeCache.txt  CMakeFiles  cmake_install.cmake
</code></pre>

Wesley_Neill: <p>It looks like it was expecting a filename, not a directory. Once provided with the full path to script.h, it worked… kind of.</p>
<p>Now it can’t find the includes listed at the top of script.h and providing a second file location in the command line argument does not seem to remedy things.</p>

Wesley_Neill: <p>Rather than trying to compile with nvcc directly, I decided to try using <code>torch::RegisterOperators</code> and <code>torch.cpp_extension</code> to take care of compiling the cuda file in the background. That worked and since I’m specifically looking to load custom pytorch operators was good enough. I’d like to learn more about general nvcc compilation some day.</p>