CUDA configuration with libtorch C++ and training time control for CNN model

Amadou: <p>Hello,<br>
I am trying to implement a CNN model with libtorch C++ on WSL2. I have:</p>
<ul>
<li>NVIDIA T1200 Laptop GPU with 4GiB on GPU</li>
<li>12 cores CPU</li>
<li>installed <code>Cudatoolkit version 11.7</code></li>
<li>installed <code>CuDNN version 8.9.1.23</code></li>
<li>installed <code>Pytorch 1.13.1+cu117</code></li>
</ul>
<p>Then I used  C++ API to install Torch in C++ using <code>Torch_DIR='python -c "import torch;print(torch.utils.cmake_prefix_path)" '</code>.</p>
<p>After building the executable with <code>cmake ..</code> I get the following result with 2 warnings :</p>
<blockquote>
<p>– The C compiler identification is GNU 11.4.0<br>
– The CXX compiler identification is GNU 11.4.0<br>
– Detecting C compiler ABI info<br>
– Detecting C compiler ABI info - done<br>
– Check for working C compiler: /usr/bin/cc - skipped<br>
– Detecting C compile features<br>
– Detecting C compile features - done<br>
– Detecting CXX compiler ABI info<br>
– Detecting CXX compiler ABI info - done<br>
– Check for working CXX compiler: /usr/bin/c++ - skipped<br>
– Detecting CXX compile features<br>
– Detecting CXX compile features - done<br>
– Looking for pthread.h<br>
– Looking for pthread.h - found<br>
– Performing Test CMAKE_HAVE_LIBC_PTHREAD<br>
– Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success<br>
– Found Threads: TRUE<br>
– Found CUDA: /usr/local/cuda-11.7 (found version “11.7”)<br>
– Found CUDA: /usr/local/cuda-11.7 (found version “11.7”)<br>
– The CUDA compiler identification is NVIDIA 11.5.119<br>
– Detecting CUDA compiler ABI info<br>
– Detecting CUDA compiler ABI info - done<br>
– Check for working CUDA compiler: /usr/bin/nvcc - skipped<br>
– Detecting CUDA compile features<br>
– Detecting CUDA compile features - done<br>
– Caffe2: CUDA detected: 11.7<br>
– Caffe2: CUDA nvcc is: /usr/local/cuda-11.7/bin/nvcc<br>
– Caffe2: CUDA toolkit directory: /usr/local/cuda-11.7<br>
– Caffe2: Header version is: 11.7<br>
– Found CUDNN: /usr/local/cuda-11.7/lib64/libcudnn.so<br>
– Found cuDNN: v8.9.1  (include: /usr/local/cuda-11.7/include, library: /usr/local/cuda-11.7/lib64/libcudnn.so)<br>
– /usr/local/cuda-11.7/lib64/libnvrtc.so shorthash is d833c4f3<br>
– Autodetected CUDA architecture(s):  7.5<br>
– Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75<br>
CMake Warning at /home/amadou/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):<br>
static library kineto_LIBRARY-NOTFOUND not found.<br>
Call Stack (most recent call first):<br>
/home/amadou/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)<br>
CMakeLists.txt:29 (find_package)<br>
– Found Torch: /home/amadou/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/lib/libtorch.so<br>
– Configuring done<br>
CMake Warning at CMakeLists.txt:37 (add_executable):<br>
Cannot generate a safe runtime search path for target cyconv because files<br>
in some directories may conflict with libraries in implicit directories:</p>
</blockquote>
<p>runtime library [libnvToolsExt.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:<br>
/usr/local/cuda-11.7/lib64</p>
<p>Some of these libraries may not be found correctly.</p>
<p>– Generating done<br>
– Build files have been written to: /home/amadou/Development/cyconv_with_pytorch/build</p>
<p>Then I do <code>make</code> and train my CNN model. But what bothers me is the execution time on GPU, which is too high. In fact, it’s not as fast as when I run the model on CPU (i.e without using CUDA).</p>
<p>I suspect a CUDA &amp; Torch configuration problem on C++. I don’t have any errors in my code, but it’s just the training time that’s too high.</p>
<p>I can’t work out why I’m not actually using the power of GPU. Can you help me find the reason?</p>
<p>Thanks in advance !</p>