PyTorch crashes when running with OpenACC

estojoverde: <p>I already opened an issue (<a href="https://github.com/pytorch/pytorch/issues/82627" class="inline-onebox" rel="noopener nofollow ugc">PyTorch crashes when running with OpenACC · Issue #82627 · pytorch/pytorch · GitHub</a>), but I’m hopping the community could also give me pointers on what I could do to start debugging it.</p>
<hr>
<h3>
<a name="describe-the-bug-1" class="anchor" href="#describe-the-bug-1"></a><img src="https://discuss.pytorch.org/images/emoji/apple/bug.png?v=12" title=":bug:" class="emoji" alt=":bug:" loading="lazy" width="20" height="20"> Describe the bug</h3>
<p>I’m binding OpenACC code with ctypes, and it is working fine. However, just by importing torch pkg, it crashes the application.</p>
<p>module_c.cpp</p>
<pre><code class="lang-auto">#include "module_c.h"

int addvector_cab(void)
{
    
    int i;
    float a[50];
    float b[50];
    float c[50];
    int n=50;

    for( i=0; i&lt;n; i++)
    {
        a[i] = 1;
        b[i] = 1;
        c[i] = 0;
    }
    
    printf("ENTERED C FUNCTION!\n");

    if( n == 0 ){
        printf("DUMMY ERROR!\n");
        printf("EXITING C FUNCTION!\n");
        return(1);
    }
    
    #pragma acc parallel loop present_or_copyin(a,b) present_or_copyout(c)
    for(i = 0; i &lt; n; i++){
        c[i] = a[i] + b[i];
    }

    printf("EXITING C FUNCTION!\n");
    return(0);
}

</code></pre>
<p>module_c.h :</p>
<pre><code class="lang-auto">#pragma once

#ifndef __MODULE_C_H_INCLUDED__
#define __MODULE_C_H_INCLUDED__

#include &lt;iostream&gt;
#include &lt;string&gt;
#include "openacc.h"
#include "stdlib.h"


extern "C" {

    int addvector_cab(void);
}

#endif

</code></pre>
<p>Compiling lines:</p>
<pre><code class="lang-auto">nvc++ -c -std=c++11 -acc -ta=multicore -fPIC -o module_c.o  module_c.cpp
nvc++ -shared -Minfo=acc -std=c++11 -mp -acc:gpu -gpu=pinned   -o mylib.so  module_c.o

</code></pre>
<p>bind.py :</p>
<pre><code class="lang-auto">import ctypes
#import torch

so_file = "./mylib.so"

my_functions = ctypes.CDLL(so_file)

my_functions.addvector_cab.restype = ctypes.c_int

if( my_functions.addvector_cab() == 0):
    print("Returned OKAY!")
</code></pre>
<h2>
<a name="expected-outputs-2" class="anchor" href="#expected-outputs-2"></a>Expected Outputs</h2>
<p>One should expect:</p>
<pre><code class="lang-auto">ENTERED C FUNCTION!
EXITING C FUNCTION!
Returned OKAY!
</code></pre>
<p>However, importing PyTorch in bind.py (uncommeting line 2, nothing else changed) and running again, it returns:</p>
<pre><code class="lang-auto">ENTERED C FUNCTION!

libgomp: TODO
</code></pre>
<p>Not sure if is related, but I tried a similar approach with libtorch in C++, and whenever I tried to run a code with OpenACC and libtorch, same thing happened… it just crashed and output ‘libgomp: TODO’.</p>
<p>What I’m trying behind all this is to allocate a tensor via torch, share it with Cupy via Cuda_Array_Interface, and them use it in OpenACC (I’m already doing this last part without errors, if I allocated memory via Cupy). But the error I’m getting is way more basic than that… just by import torch, it crashes.</p>
<p>Any help/hint/axes are appreciated. =]</p>
<p>EDIT: Due to space constraints, I’ve simplified some parts… better documentation and example can be found here: <a href="https://github.com/estojoverde/Torch_OpenACC/blob/pytorch_openacc" rel="noopener nofollow ugc">https://github.com/estojoverde/Torch_OpenACC/blob/pytorch_openacc</a></p>
<h3>
<a name="versions-3" class="anchor" href="#versions-3"></a>Versions</h3>
<p>Collecting environment information…<br>
PyTorch version: 1.12.0<br>
Is debug build: False<br>
CUDA used to build PyTorch: 11.6<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 20.04.4 LTS (x86_64)<br>
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0<br>
Clang version: Could not collect<br>
CMake version: version 3.22.3<br>
Libc version: glibc-2.31</p>
<p>Python version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:18)  [GCC 10.3.0] (64-bit runtime)<br>
Python platform: Linux-3.10.0-1160.49.1.el7.x86_64-x86_64-with-glibc2.10<br>
Is CUDA available: True<br>
CUDA runtime version: 11.6.124<br>
GPU models and configuration:<br>
GPU 0: Tesla V100-PCIE-32GB<br>
GPU 1: Tesla V100-PCIE-32GB<br>
GPU 2: Tesla V100-PCIE-32GB</p>
<p>Nvidia driver version: 510.47.03<br>
cuDNN version: Probably one of the following:<br>
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.0<br>
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.0<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A<br>
Is XNNPACK available: True</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.22.3<br>
[pip3] torch==1.12.0<br>
[pip3] torchaudio==0.12.0<br>
[pip3] torchvision==0.13.0<br>
[conda] blas                      1.0                         mkl    anaconda<br>
[conda] cudatoolkit               11.6.0              hecad31d_10    conda-forge<br>
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch<br>
[conda] mkl                       2021.4.0           h06a4308_640    anaconda<br>
[conda] mkl-service               2.4.0            py38h95df7f1_0    conda-forge<br>
[conda] mkl_fft                   1.3.1            py38h8666266_1    conda-forge<br>
[conda] mkl_random                1.2.2            py38h1abd341_0    conda-forge<br>
[conda] numpy                     1.22.3           py38he7a7128_0    anaconda<br>
[conda] numpy-base                1.22.3           py38hf524024_0    anaconda<br>
[conda] pytorch                   1.12.0          py3.8_cuda11.6_cudnn8.3.2_0    pytorch<br>
[conda] pytorch-mutex             1.0                        cuda    pytorch<br>
[conda] torchaudio                0.12.0               py38_cu116    pytorch<br>
[conda] torchvision               0.13.0               py38_cu116    pytorch</p>