Building LibTorch without CUDA fails

Sven1: <p>I have a conda environment with all require dependencies to build the C++ LibTorch Libraries on my Ubunutu 18.04 Machine. The cmake build process with Cuda 11.1 works fine.</p>
<pre><code class="lang-auto">SOURCE_DIR=pytorch
BUILD_DIR=pytorch-cuda-build
INSTALL_DIR=pytorch-cuda-install

echo "Setup exports for Cuda 11.1..."
export CUDA_NVCC_EXECUTABLE="/usr/local/cuda-11.1/bin/nvcc"
export CUDA_HOME="/usr/local/cuda-11.1"
export CUDA_BIN_PATH="/usr/local/cuda-11.1"
export CUDA_TOOLKIT_ROOT_DIR="/usr/local/cuda-11.1"
export CUDNN_INCLUDE_PATH="/usr/local/cuda-11.1/include/"
export CUDNN_LIBRARY_PATH="/usr/local/cuda-11.1/lib64/"
export LIBRARY_PATH="/usr/local/cuda-11.1/lib64"

mkdir $BUILD_DIR
cd $BUILD_DIR

echo "Build libtorch with Cuda 11.1 support..."
export USE_CUDA=1 USE_CUDNN=1 USE_MKLDNN=1
cmake -j 8 -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_INSTALL_PREFIX:PATH=../$INSTALL_DIR ../$SOURCE_DIR
</code></pre>
<p>Now I need compile the LibTorch Libraries without the CUDA envrionment. Therefore I changed the script.</p>
<pre><code class="lang-auto">echo "Build libtorch without Cuda support..."
export USE_CUDA=0 USE_CUDNN=0
cmake -j 8 -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_INSTALL_PREFIX:PATH=../$INSTALL_DIR ../$SOURCE_DIR
</code></pre>
<p>The compilation fails with the following error message. It looks like the scripts still search for any CUDA libraries. Any Ideas?</p>
<pre><code class="lang-auto">Build libtorch without Cuda...
-- The CXX compiler identification is GNU 8.4.0
-- The C compiler identification is GNU 8.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Not forcing any particular BLAS to be found
-- Performing Test COMPILER_WORKS
-- Performing Test COMPILER_WORKS - Success
-- Performing Test SUPPORT_GLIBCXX_USE_C99
-- Performing Test SUPPORT_GLIBCXX_USE_C99 - Success
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Success
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS - Success
-- Current compiler supports avx2 extension. Will build perfkernels.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS - Success
-- Current compiler supports avx512f extension. Will build fbgemm.
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC - Success
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
-- 
-- 3.11.4.0
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS - Success
-- Caffe2 protobuf include directory: $&lt;BUILD_INTERFACE:/home/dd/workspace/AI_EnvPerception/frameworks/pytorch/third_party/protobuf/src&gt;$&lt;INSTALL_INTERFACE:include&gt;
-- Trying to find preferred BLAS backend of choice: MKL
-- MKL_THREADING = OMP
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_C)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  cmake/Modules/FindMKL.cmake:213 (FIND_PACKAGE)
  cmake/Modules/FindMKL.cmake:307 (CHECK_ALL_LIBRARIES)
  cmake/Dependencies.cmake:144 (find_package)
  CMakeLists.txt:621 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  cmake/Modules/FindMKL.cmake:213 (FIND_PACKAGE)
  cmake/Modules/FindMKL.cmake:307 (CHECK_ALL_LIBRARIES)
  cmake/Dependencies.cmake:144 (find_package)
  CMakeLists.txt:621 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Looking for cblas_sgemm
-- Looking for cblas_sgemm - found
-- MKL libraries: /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_gnu_thread.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;-fopenmp;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.so
-- MKL include directory: /opt/intel/mkl/include
-- MKL OpenMP type: GNU
-- MKL OpenMP library: -fopenmp
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Brace yourself, we are building NNPACK
-- Performing Test NNPACK_ARCH_IS_X86_32
-- Performing Test NNPACK_ARCH_IS_X86_32 - Failed
-- Found PythonInterp: /home/dd/miniconda3/envs/pytorch-build/bin/python3 (found version "3.6.13") 
-- NNPACK backend is x86-64
-- Performing Test COMPILER_SUPPORTS_AVX512
-- Performing Test COMPILER_SUPPORTS_AVX512 - Success
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_C)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  third_party/fbgemm/CMakeLists.txt:60 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenMP_C: -fopenmp (found version "4.5") 
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  third_party/fbgemm/CMakeLists.txt:60 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
CMake Warning at third_party/fbgemm/CMakeLists.txt:62 (message):
  OpenMP found! OpenMP_C_INCLUDE_DIRS =


CMake Warning at third_party/fbgemm/CMakeLists.txt:142 (message):
  ==========


CMake Warning at third_party/fbgemm/CMakeLists.txt:143 (message):
  CMAKE_BUILD_TYPE = Release


CMake Warning at third_party/fbgemm/CMakeLists.txt:144 (message):
  CMAKE_CXX_FLAGS_DEBUG is -g


CMake Warning at third_party/fbgemm/CMakeLists.txt:145 (message):
  CMAKE_CXX_FLAGS_RELEASE is -O3 -DNDEBUG


CMake Warning at third_party/fbgemm/CMakeLists.txt:146 (message):
  ==========


-- Performing Test __CxxFlag__fno_threadsafe_statics
-- Performing Test __CxxFlag__fno_threadsafe_statics - Success
-- Performing Test __CxxFlag__fno_semantic_interposition
-- Performing Test __CxxFlag__fno_semantic_interposition - Success
-- Performing Test __CxxFlag__fmerge_all_constants
-- Performing Test __CxxFlag__fmerge_all_constants - Success
-- Performing Test __CxxFlag__fno_enforce_eh_specs
-- Performing Test __CxxFlag__fno_enforce_eh_specs - Success
** AsmJit Summary **
   ASMJIT_DIR=/home/dd/workspace/AI_EnvPerception/frameworks/pytorch/third_party/fbgemm/third_party/asmjit
   ASMJIT_TEST=FALSE
   ASMJIT_TARGET_TYPE=STATIC
   ASMJIT_DEPS=pthread;rt
   ASMJIT_LIBS=asmjit;pthread;rt
   ASMJIT_CFLAGS=-DASMJIT_STATIC
   ASMJIT_PRIVATE_CFLAGS=-Wall;-Wextra;-Wconversion;-fno-math-errno;-fno-threadsafe-statics;-fno-semantic-interposition;-DASMJIT_STATIC
   ASMJIT_PRIVATE_CFLAGS_DBG=
   ASMJIT_PRIVATE_CFLAGS_REL=-O2;-fmerge-all-constants;-fno-enforce-eh-specs
-- Found Numa: /usr/include  
-- Found Numa  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
-- Using third party subdirectory Eigen.
Python 3.6.13 :: Anaconda, Inc.
-- Setting Python's include dir to /home/dd/miniconda3/envs/pytorch-build/include/python3.6m from sysconfig
-- Setting Python's library to /home/dd/miniconda3/envs/pytorch-build/lib/python3.6
-- Found PythonInterp: /home/dd/miniconda3/envs/pytorch-build/bin/python3 (found suitable version "3.6.13", minimum required is "3.0") 
-- Found PythonLibs: /home/dd/miniconda3/envs/pytorch-build/lib/python3.6 (found suitable version "3.6.13", minimum required is "3.0") 
-- Found NumPy: /home/dd/.local/lib/python3.6/site-packages/numpy/core/include (found version "1.19.5") 
-- NumPy ver. 1.19.5 found (include: /home/dd/.local/lib/python3.6/site-packages/numpy/core/include)
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR) 
-- Using third_party/pybind11.
-- pybind11 include dirs: /home/dd/workspace/AI_EnvPerception/frameworks/pytorch/cmake/../third_party/pybind11/include
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- MPI support found
-- MPI compile flags: -pthread
-- MPI include path: /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include/usr/lib/x86_64-linux-gnu/openmpi/include
-- MPI LINK flags path: -L/usr/lib -pthread
-- MPI libraries: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so
CMake Warning at cmake/Dependencies.cmake:1045 (message):
  OpenMPI found, but it is not built with CUDA support.
Call Stack (most recent call first):
  CMakeLists.txt:621 (include)


CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_C)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  cmake/Dependencies.cmake:1105 (find_package)
  CMakeLists.txt:621 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
  The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
  does not match the name of the calling package (OpenMP).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
  cmake/Dependencies.cmake:1105 (find_package)
  CMakeLists.txt:621 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Adding OpenMP CXX_FLAGS: -fopenmp
-- Will link against OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/8/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Found CUDA: /usr (found version "9.1") 
-- Caffe2: CUDA detected: 9.1
-- Caffe2: CUDA nvcc is: /usr/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr
-- Caffe2: Header version is: 9.1
-- Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so  
-- Found cuDNN: v8.2.4  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- /usr/lib/x86_64-linux-gnu/libnvrtc.so shorthash is 9862b2bf
CMake Error at cmake/public/cuda.cmake:440 (message):
  CUDA 9.1 is not compatible with std::tuple from GCC version &gt;= 6.  Please
  upgrade to CUDA 9.2 or set the following environment variable to use
  another version (for example):

    export CUDAHOSTCXX='/usr/bin/gcc-5'

Call Stack (most recent call first):
  cmake/Dependencies.cmake:1155 (include)
  CMakeLists.txt:621 (include)


-- Configuring incomplete, errors occurred!
See also "/home/dd/workspace/AI_EnvPerception/frameworks/pytorch-cpu-build/CMakeFiles/CMakeOutput.log".
See also "/home/dd/workspace/AI_EnvPerception/frameworks/pytorch-cpu-build/CMakeFiles/CMakeError.log".
make: *** No rule to make target 'install'.  Stop.
Build done! Check pytorch-cpu-build folder for libraries.
</code></pre>

ptrblck: <p>PyTorch needs CUDA&gt;=10.2 right now while you are using 9.1, which is thus breaking with:</p>
<pre><code class="lang-python">CMake Error at cmake/public/cuda.cmake:440 (message):
  CUDA 9.1 is not compatible with std::tuple from GCC version &gt;= 6.  Please
  upgrade to CUDA 9.2 or set the following environment variable to use
  another version (for example)
</code></pre>

Sven1: <p>I’m wondering why the build process is still searching for CUDA. I explicitly defined to build without CUDA support.</p>
<pre><code class="lang-auto">export USE_CUDA=0 USE_CUDNN=0
</code></pre>

ptrblck: <p>Did you clean the build after the last run? I’m able to use <code>USE_CUDA=0</code> in my environment with an installed CUDA toolkit.</p>

Sven1: <p>Yes, I delete all my install and build directories before compiling the library. I even started a new terminal to have a clean environment.</p>
<p>It seems that my <code>export USE_CUDA=0 USE_CUDNN=0</code> have no effect. <code>USE_CUDA</code> is set to ON in the log summary.</p>
<pre><code class="lang-auto">--   USE_CUDA              : ON
--     Split CUDA          : OFF
--     CUDA static link    : OFF
--     USE_CUDNN           : ON
--     CUDA version        : 11.1
--     cuDNN version       : 8.2.4
--     CUDA root directory : /usr/local/cuda-11.1
--     CUDA library        : /usr/local/cuda-11.1/lib64/stubs/libcuda.so
--     cudart library      : /usr/local/cuda-11.1/lib64/libcudart.so
--     cublas library      : /usr/local/cuda-11.1/lib64/libcublas.so
--     cufft library       : /usr/local/cuda-11.1/lib64/libcufft.so
--     curand library      : /usr/local/cuda-11.1/lib64/libcurand.so
--     cuDNN library       : /usr/lib/x86_64-linux-gnu/libcudnn.so
--     nvrtc               : /usr/local/cuda-11.1/lib64/libnvrtc.so
--     CUDA include path   : /usr/local/cuda-11.1/include
--     NVCC executable     : /usr/local/cuda-11.1/bin/nvcc
</code></pre>
<p>Do I need to set any further parameters?</p>

Sven1: <p>The export is ignored by cmake. I solved it by adding the <code>-DUSE_CUDA=OFF</code> parameter to the cmake command.</p>
<p><code>cmake -j 8 -DUSE_CUDA=OFF -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=</code>which python3<code> -DCMAKE_INSTALL_PREFIX:PATH=../$INSTALL_DIR ../$SOURCE_DIR</code></p>