Cannot build Pytorch from source

rohitdat: <p>Hey guys, I am experimenting with MPI and mpi4py to parallelize on multiple processes/single machine (linux) a basic neural network.<br>
I understand that PyTorch should be compiled from source when using MPI backend so I followed the guidelines and when doing</p>
<pre><code class="lang-auto">python setup.py install
</code></pre>
<p>I get the following terminal error and I cannot figure out a solution:</p>
<pre><code class="lang-auto">Building wheel torch-2.2.0a0+git3cf5348
-- Building version 2.2.0a0+git3cf5348
cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/rohitkumar/github_code/distributed-neural-network/pytorch/torch -DCMAKE_PREFIX_PATH=/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages;/home/rohitkumar/.conda/envs/december -DGLIBCXX_USE_CXX11_ABI=1 -DNUMPY_INCLUDE_DIR=/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/home/rohitkumar/.conda/envs/december/bin/python3 -DPYTHON_INCLUDE_DIR=/home/rohitkumar/.conda/envs/december/include/python3.10 -DPYTHON_LIBRARY=/home/rohitkumar/.conda/envs/december/lib/libpython3.10.a -DTORCH_BUILD_VERSION=2.2.0a0+git3cf5348 -DUSE_NUMPY=True /home/rohitkumar/github_code/distributed-neural-network/pytorch
CMake Error: Error: generator : Ninja
Does not match the generator used previously: Unix Makefiles
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
</code></pre>
<p>When checking in the cloned ‘pytorch’ directory, I cannot find any CMakeCache.txt file or any CMakeFiles directory.</p>
<p>Any help would be highly appreciated</p>
<p>Thanks!</p>

ptrblck: <p>Could you clean the build environment via <code>python setup.py clean</code> and rerun the build step?</p>

rohitdat: <p>I am not sure what you refer to when saying “build step”. Could you specify?</p>
<p>I ran without any error:</p>
<pre><code class="lang-auto">python setup.py clean
</code></pre>
<p>Then ran:</p>
<pre><code class="lang-auto">python setup.py install
</code></pre>
<p>Which still gives me the same error</p>

rohitdat: <p>hey <a class="mention" href="/u/ptrblck">@ptrblck</a> your guidance would extremely help me. Thank you in advance!</p>

ptrblck: <p>No, as I don’t see the actual error. Could you check the full build log to see if anything raised a proper error message?</p>

rohitdat: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> So I deleted the build folder, CMakeFiles folder, CMakeCache.txt file.<br>
I recreated a build folder from where I ran the following command:</p>
<pre><code class="lang-auto">cmake -GNinja     
-DBUILD_PYTHON=True    
 -DBUILD_TEST=True     
-DCMAKE_BUILD_TYPE=Release     
-DCMAKE_INSTALL_PREFIX=/home/rohitkumar/github_code/distributed-neural-network/pytorch/torch   
 -DCMAKE_PREFIX_PATH="/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages;/home/rohitkumar/.conda/envs/december"    
 -DNUMPY_INCLUDE_DIR="/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages/numpy/core/include"    
 -DPYTHON_EXECUTABLE=/home/rohitkumar/.conda/envs/december/bin/python     
-DPYTHON_INCLUDE_DIR=/home/rohitkumar/.conda/envs/december/include/python3.10     
-DPYTHON_LIBRARY=/home/rohitkumar/.conda/envs/december/lib/libpython3.10.a     
-DTORCH_BUILD_VERSION=2.2.0a0+git3cf5348    
 -DUSE_NUMPY=True    
 ..
</code></pre>
<p>Which throws this error (I dumped the full log here: <a href="https://gist.github.com/dat-rohit/3308d047d8f1b430961cce76d142eb6c" class="inline-onebox" rel="noopener nofollow ugc">gist:3308d047d8f1b430961cce76d142eb6c · GitHub</a>):</p>
<pre><code class="lang-auto">
-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.26.4
--   CMake command         : /home/rohitkumar/.conda/envs/december/bin/cmake
--   System                : Linux
--   C++ compiler          : /opt/ohpc/pub/compiler/gcc/7.3.0/bin/c++
--   C++ compiler id       : GNU
--   C++ compiler version  : 7.3.0
--   Using ccache if found : ON
--   Found ccache          : CCACHE_PROGRAM-NOTFOUND
--   CXX flags             :  -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow
--   Build type            : Release
--   Compile definitions   : ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;IDEEP_USE_MKL;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS
--   CMAKE_PREFIX_PATH     : /home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages;/home/rohitkumar/.conda/envs/december
--   CMAKE_INSTALL_PREFIX  : /home/rohitkumar/github_code/distributed-neural-network/pytorch/torch
--   USE_GOLD_LINKER       : OFF
-- 
--   TORCH_VERSION         : 2.2.0
--   BUILD_CAFFE2          : OFF
--   BUILD_CAFFE2_OPS      : OFF
--   BUILD_STATIC_RUNTIME_BENCHMARK: OFF
--   BUILD_TENSOREXPR_BENCHMARK: OFF
--   BUILD_BINARY          : OFF
--   BUILD_CUSTOM_PROTOBUF : ON
--     Link local protobuf : ON
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : True
--     Python version      : 3.10.13
--     Python executable   : /home/rohitkumar/.conda/envs/december/bin/python
--     Pythonlibs version  : 3.10.13
--     Python library      : /home/rohitkumar/.conda/envs/december/lib/libpython3.10.a
--     Python includes     : /home/rohitkumar/.conda/envs/december/include/python3.10
--     Python site-packages: lib/python3.10/site-packages
--   BUILD_SHARED_LIBS     : ON
--   CAFFE2_USE_MSVC_STATIC_RUNTIME     : OFF
--   BUILD_TEST            : True
--   BUILD_JNI             : OFF
--   BUILD_MOBILE_AUTOGRAD : OFF
--   BUILD_LITE_INTERPRETER: OFF
--   INTERN_BUILD_MOBILE   : 
--   TRACING_BASED         : OFF
--   USE_BLAS              : 1
--     BLAS                : mkl
--     BLAS_HAS_SBGEMM     : 
--   USE_LAPACK            : 1
--     LAPACK              : mkl
--   USE_ASAN              : OFF
--   USE_TSAN              : OFF
--   USE_CPP_CODE_COVERAGE : OFF
--   USE_CUDA              : OFF
--   USE_ROCM              : OFF
--   BUILD_NVFUSER         : 
--   USE_EIGEN_FOR_BLAS    : 
--   USE_FBGEMM            : ON
--     USE_FAKELOWP          : OFF
--   USE_KINETO            : ON
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : OFF
--   USE_GLOG              : OFF
--   USE_LEVELDB           : OFF
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : OFF
--   USE_METAL             : OFF
--   USE_PYTORCH_METAL     : OFF
--   USE_PYTORCH_METAL_EXPORT     : OFF
--   USE_MPS               : OFF
--   USE_MKL               : ON
--   USE_MKLDNN            : ON
--   USE_MKLDNN_ACL        : OFF
--   USE_MKLDNN_CBLAS      : OFF
--   USE_UCC               : OFF
--   USE_ITT               : ON
--   USE_NCCL              : OFF
--   USE_NNPACK            : ON
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCL            : OFF
--   USE_OPENCV            : OFF
--   USE_OPENMP            : ON
--   USE_TBB               : OFF
--   USE_MIMALLOC          : OFF
--   USE_VULKAN            : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : ON
--   USE_PYTORCH_QNNPACK   : ON
--   USE_XNNPACK           : ON
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : ON
--     USE_MPI               : ON
--     USE_GLOO              : ON
--     USE_GLOO_WITH_OPENSSL : OFF
--     USE_TENSORPIPE        : ON
--   Public Dependencies  : caffe2::mkl
--   Private Dependencies : Threads::Threads;pthreadpool;cpuinfo;qnnpack;pytorch_qnnpack;nnpack;XNNPACK;fbgemm;ittnotify;fp16;caffe2::openmp;tensorpipe;gloo;foxi_loader;rt;fmt::fmt-header-only;kineto;gcc_s;gcc;dl
--   Public CUDA Deps.    : 
--   Private CUDA Deps.   : 
--   USE_COREML_DELEGATE     : OFF
--   BUILD_LAZY_TS_BACKEND   : ON
--   TORCH_DISABLE_GPU_ASSERTS : ON
-- Configuring done (28.6s)
-- Generating done (4.4s)
CMake Error:
  Running

   '/home/rohitkumar/.conda/envs/december/bin/ninja' '-C' '/home/rohitkumar/github_code/distributed-neural-network/pytorch/build' '-t' 'recompact'

  failed with:

   ninja: error: build.ninja:48678: multiple rules generate caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/UfuncCPUKernel_add.cpp.DEFAULT.cpp.o [-w dupbuild=err]

  



CMake Generate step failed.  Build files cannot be regenerated correctly.
</code></pre>
<p>Regardless of this error, I tried running the following from main folder:</p>
<pre><code class="lang-auto">python setup.py install
</code></pre>
<p>Which gave me this output:</p>
<pre><code class="lang-auto">Building wheel torch-2.2.0a0+git3cf5348
-- Building version 2.2.0a0+git3cf5348
cmake --build . --target install --config Release
ninja: error: build.ninja:48678: multiple rules generate caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/UfuncCPUKernel_add.cpp.DEFAULT.cpp.o [-w dupbuild=err]
</code></pre>
<p>BTW, I am on VSCODE and have CUDA Version: 11.0<br>
Please tell me if any other information or step would help the resolution</p>
<p>Thanks!</p>

ptrblck: <p>Thanks for posting the full log. I haven’t seen this error before:</p>
<pre data-code-wrap="python"><code class="lang-python"> ninja: error: build.ninja:48678: multiple rules generate caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/UfuncCPUKernel_add.cpp.DEFAULT.cpp.o [-w dupbuild=err]
</code></pre>
<p>but also see that you are using an older C++ toolchain:</p>
<pre><code class="lang-auto">--   C++ compiler          : /opt/ohpc/pub/compiler/gcc/7.3.0/bin/c++
--   C++ compiler id       : GNU
--   C++ compiler version  : 7.3.0
</code></pre>
<p>Could you update to gcc 9.4+ and see if this would fix the build issue?</p>

rohitdat: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you for your quick answer and sorry for the late reply, I finally managed to update the gcc version used by CMake to version 11.2.0.</p>
<p>Then again, I ran</p>
<pre><code class="lang-auto">cmake -GNinja     
-DBUILD_PYTHON=True    
 -DBUILD_TEST=True     
-DCMAKE_BUILD_TYPE=Release     
-DCMAKE_INSTALL_PREFIX=/home/rohitkumar/github_code/distributed-neural-network/pytorch/torch   
 -DCMAKE_PREFIX_PATH="/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages;/home/rohitkumar/.conda/envs/december"    
 -DNUMPY_INCLUDE_DIR="/home/rohitkumar/.conda/envs/december/lib/python3.10/site-packages/numpy/core/include"    
 -DPYTHON_EXECUTABLE=/home/rohitkumar/.conda/envs/december/bin/python     
-DPYTHON_INCLUDE_DIR=/home/rohitkumar/.conda/envs/december/include/python3.10     
-DPYTHON_LIBRARY=/home/rohitkumar/.conda/envs/december/lib/libpython3.10.a     
-DTORCH_BUILD_VERSION=2.2.0a0+git3cf5348    
 -DUSE_NUMPY=True    
 ..

</code></pre>
<p>Which now gives me this error:</p>
<pre><code class="lang-auto">CMake Warning at CMakeLists.txt:36 (message):
  C++ standard version definition detected in environment variable.PyTorch
  requires -std=c++17.  Please remove -std=c++ settings in your environment.


-- /home/rohitkumar/.conda/envs/december/bin/x86_64-conda-linux-gnu-g++ /home/rohitkumar/github_code/distributed-neural-network/pytorch/torch/abi-check.cpp -o /home/rohitkumar/github_code/distributed-neural-network/pytorch/build/abi-check
/home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/lib/../lib64/libstdc++.so: undefined reference to `memcpy@GLIBC_2.14'
/home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/lib/../lib64/libstdc++.so: undefined reference to `aligned_alloc@GLIBC_2.16'
/home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/rohitkumar/.conda/envs/december/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/lib/../lib64/libstdc++.so: undefined reference to `clock_gettime@GLIBC_2.17'
collect2: error: ld returned 1 exit status
CMake Error at cmake/CheckAbi.cmake:16 (message):
  Could not compile ABI Check: 1
Call Stack (most recent call first):
  CMakeLists.txt:52 (include)
</code></pre>
<p>I tried adding the following flags to the cmake command:</p>
<pre><code class="lang-auto">-DCMAKE_CXX_STANDARD=17  -DCMAKE_CXX_STANDARD_REQUIRED=ON
</code></pre>
<p>However, it still gives me the same error.</p>
<p>Regardless, I tried cleaning the build environment and rerun python setup.py install, which now gives me this error (this is only the end of the log): <a href="https://gist.github.com/dat-rohit/24576bfca6f8b44bb87d7e21e7eacd2b" class="inline-onebox" rel="noopener nofollow ugc">gist:24576bfca6f8b44bb87d7e21e7eacd2b · GitHub</a></p>
<p>Please tell me if there is anything else I can try, I am clueless, thanks!</p>

rprabhu: <p>Hey, I’m also facing the “aten/src/ATen/UfuncCPUKernel_add.cpp.DEFAULT.cpp.o [-w dupbuild=err]” issue. did you guys manage to find a solution <a class="mention" href="/u/rohitdat">@rohitdat</a>  <a class="mention" href="/u/ptrblck">@ptrblck</a> ?</p>

ptrblck: <p>I don’t know what kind of error you are seeing. The last post in this topic pointed to a C++ toolchain issue given GLIBC symbols were undefined.</p>

rprabhu: <p>when I build pytorch inside the pytorch/pytorch:2.1.0-cuda12.1-cudnn8-devel, [i cloned the v2.1.0 version], it gives me the following error:</p>
<pre><code class="lang-auto">
-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.26.4
--   CMake command         : /opt/conda/envs/nano_paging/bin/cmake
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler id       : GNU
--   C++ compiler version  : 9.4.0
--   Using ccache if found : ON
--   Found ccache          : CCACHE_PROGRAM-NOTFOUND
--   CXX flags             :  -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-unused-private-field -Wno-aligned-allocation-unavailable -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow
--   Build type            : Release
--   Compile definitions   : ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;IDEEP_USE_MKL;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS;BUILD_NVFUSER
--   CMAKE_PREFIX_PATH     : /opt/conda/envs/nano_paging/lib/python3.10/site-packages;/opt/conda/envs/nano_paging;/usr/local/cuda;/usr/local/cuda
--   CMAKE_INSTALL_PREFIX  : /ramyapra/pytorch/torch
--   USE_GOLD_LINKER       : OFF
-- 
--   TORCH_VERSION         : 2.1.0
--   BUILD_CAFFE2          : 0
--   BUILD_CAFFE2_OPS      : OFF
--   BUILD_STATIC_RUNTIME_BENCHMARK: OFF
--   BUILD_TENSOREXPR_BENCHMARK: OFF
--   BUILD_NVFUSER_BENCHMARK: OFF
--   BUILD_BINARY          : OFF
--   BUILD_CUSTOM_PROTOBUF : ON
--     Link local protobuf : ON
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : True
--     Python version      : 3.10.13
--     Python executable   : /opt/conda/envs/nano_paging/bin/python
--     Pythonlibs version  : 3.10.13
--     Python library      : /opt/conda/envs/nano_paging/lib/libpython3.10.a
--     Python includes     : /opt/conda/envs/nano_paging/include/python3.10
--     Python site-packages: lib/python3.10/site-packages
--   BUILD_SHARED_LIBS     : ON
--   CAFFE2_USE_MSVC_STATIC_RUNTIME     : OFF
--   BUILD_TEST            : False
--   BUILD_JNI             : OFF
--   BUILD_MOBILE_AUTOGRAD : OFF
--   BUILD_LITE_INTERPRETER: OFF
--   INTERN_BUILD_MOBILE   : 
--   TRACING_BASED         : OFF
--   USE_BLAS              : 1
--     BLAS                : mkl
--     BLAS_HAS_SBGEMM     : 
--   USE_LAPACK            : 1
--     LAPACK              : mkl
--   USE_ASAN              : OFF
--   USE_TSAN              : OFF
--   USE_CPP_CODE_COVERAGE : OFF
--   USE_CUDA              : ON
--     Split CUDA          : 
--     CUDA static link    : OFF
--     USE_CUDNN           : ON
--     USE_EXPERIMENTAL_CUDNN_V8_API: ON
--     USE_CUSPARSELT      : OFF
--     CUDA version        : 12.1
--     USE_FLASH_ATTENTION : ON
--     cuDNN version       : 8.9.0
--     CUDA root directory : /usr/local/cuda
--     CUDA library        : /usr/lib/x86_64-linux-gnu/libcuda.so
--     cudart library      : /usr/local/cuda/lib64/libcudart.so
--     cublas library      : /usr/local/cuda/lib64/libcublas.so
--     cufft library       : /usr/local/cuda/lib64/libcufft.so
--     curand library      : /usr/local/cuda/lib64/libcurand.so
--     cusparse library    : /usr/local/cuda/lib64/libcusparse.so
--     cuDNN library       : /usr/lib/x86_64-linux-gnu/libcudnn.so
--     nvrtc               : /usr/local/cuda/lib64/libnvrtc.so
--     CUDA include path   : /usr/local/cuda/include
--     NVCC executable     : /usr/local/cuda/bin/nvcc
--     CUDA compiler       : /usr/local/cuda/bin/nvcc
--     CUDA flags          :  -D_GLIBCXX_USE_CXX11_ABI=1 -Xfatbin -compress-all -DONNX_NAMESPACE=onnx_torch -gencode arch=compute_80,code=sm_80 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda  -Wno-deprecated-gpu-targets --expt-extended-lambda -DCUB_WRAPPED_NAMESPACE=at_cuda_detail -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__
--     CUDA host compiler  : 
--     CUDA --device-c     : OFF
--     USE_TENSORRT        : OFF
--   USE_ROCM              : OFF
--   BUILD_NVFUSER         : ON
--   USE_EIGEN_FOR_BLAS    : 
--   USE_FBGEMM            : ON
--     USE_FAKELOWP          : OFF
--   USE_KINETO            : ON
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : OFF
--   USE_GLOG              : OFF
--   USE_LEVELDB           : OFF
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : OFF
--   USE_METAL             : OFF
--   USE_PYTORCH_METAL     : OFF
--   USE_PYTORCH_METAL_EXPORT     : OFF
--   USE_MPS               : OFF
--   USE_FFTW              : OFF
--   USE_MKL               : ON
--   USE_MKLDNN            : ON
--   USE_MKLDNN_ACL        : OFF
--   USE_MKLDNN_CBLAS      : OFF
--   USE_UCC               : OFF
--   USE_ITT               : ON
--   USE_NCCL              : ON
--     USE_SYSTEM_NCCL     : OFF
--     USE_NCCL_WITH_UCC   : OFF
--   USE_NNPACK            : ON
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCL            : OFF
--   USE_OPENCV            : OFF
--   USE_OPENMP            : ON
--   USE_TBB               : OFF
--   USE_MIMALLOC          : OFF
--   USE_VULKAN            : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : ON
--   USE_PYTORCH_QNNPACK   : ON
--   USE_XNNPACK           : ON
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : ON
--     USE_MPI               : OFF
--     USE_GLOO              : ON
--     USE_GLOO_WITH_OPENSSL : OFF
--     USE_TENSORPIPE        : ON
--   Public Dependencies  : caffe2::mkl
--   Private Dependencies : Threads::Threads;pthreadpool;cpuinfo;qnnpack;pytorch_qnnpack;nnpack;XNNPACK;fbgemm;ittnotify;fp16;caffe2::openmp;tensorpipe;gloo;foxi_loader;rt;fmt::fmt-header-only;kineto;gcc_s;gcc;dl
--   Public CUDA Deps.    : caffe2::cufft;caffe2::curand;caffe2::cublas
--   Private CUDA Deps.   : torch::cudnn;__caffe2_nccl;tensorpipe_cuda;gloo_cuda;/usr/local/cuda/lib64/libcudart.so;CUDA::cusparse;CUDA::curand;CUDA::cufft;ATEN_CUDA_FILES_GEN_LIB
--   USE_COREML_DELEGATE     : OFF
--   BUILD_LAZY_TS_BACKEND   : ON
--   TORCH_DISABLE_GPU_ASSERTS : ON
-- Performing Test HAS_WMISSING_PROTOTYPES
-- Performing Test HAS_WMISSING_PROTOTYPES - Failed
-- Performing Test HAS_WERROR_MISSING_PROTOTYPES
-- Performing Test HAS_WERROR_MISSING_PROTOTYPES - Failed
-- Configuring done (68.2s)
-- Generating done (3.2s)
CMake Error:
  Running

   '/opt/conda/envs/nano_paging/bin/ninja' '-C' '/ramyapra/pytorch/build' '-t' 'recompact'

  failed with:

   ninja: error: build.ninja:57464: multiple rules generate caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/UfuncCPUKernel_add.cpp.DEFAULT.cpp.o [-w dupbuild=err]

  

</code></pre>
<p>i looked at the compile_commands.json file that gets generated and there indeed are two sets of commands that build this onject file, but i cannot figure out why that is happening</p>

rohitdat: <p>Hey, unfortunately after many trials, I gave up on building PyTorch from source.<br>
My purpose was to build it from source to use MPI as its backend for distributed operations but I just went for gloo backend afterwards, which doesn’t required PyTorch compiled from source to run.</p>
<p>I hope you figure out a solution for your case.</p>

GuWei007: <p>PyTorch wheel is compiled in docker manylinux:native-manylinux-builder-cpu-main. Because there is no openmpi,（USE_MPI:BOOL=OFF）. If you encounter this problem, I suggest you raise an issue in the community.<br>
Support USE_MPI:BOOL=ON, so you don’t need to Build PyTorch from source</p>

dolphin_09: <p>Similar issue I’m facing while building Pytorch with MPI backend. The issue:<br>
<a href="https://discuss.pytorch.org/t/mpi-pytorch-backend/18988" class="inline-onebox">MPI Pytorch backend</a> is exactly me issue is.</p>
<p>I would appreciate if anyone can help.</p>