Pytorch from source install fails

gakadam: <p>Set up:<br>
CUDA: 10.2<br>
GCC: 8.4.0</p>
<p>Error:</p>
<pre><code class="lang-auto">Building wheel torch-1.7.0a0+8daaa3b
-- Building version 1.7.0a0+8daaa3b
cmake --build . --target install --config Release -- -j 56
[1/5] Building CXX object c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o
FAILED: c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o
/home/gakadam/tools/gcc-8.4/bin/g++  -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMAGMA_V2 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -I../c10/.. -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /home/gakadam/tools/anaconda3/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /home/gakadam/tools/anaconda3/include/python3.8 -isystem /home/gakadam/tools/anaconda3/lib/python3.8/site-packages/numpy/core/include -isystem ../cmake/../third_party/pybind11/include -isystem ../cmake/../third_party/cub -isystem /home/gakadam/tools/cuda-10.2/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -isystem ../third_party/googletest/googlemock/include -isystem ../third_party/googletest/googlemock -isystem ../third_party/googletest/googletest/include -isystem ../third_party/googletest/googletest -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -fPIE   -DCAFFE2_USE_GLOO -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -pthread -std=gnu++14 -MD -MT c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o -MF c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o.d -o c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o -c ../c10/test/util/complex_math_test.cpp
init2.c:52: MPFR assertion failed: p &gt;= 2 &amp;&amp; p &lt;= ((mpfr_prec_t)((mpfr_uprec_t)(~(mpfr_uprec_t)0)&gt;&gt;1))
during GIMPLE pass: fre
In file included from ../c10/test/util/complex_math_test.cpp:4:
../c10/test/util/complex_math_test_common.h: In member function ‘virtual void TestRevTrigonometric_Rev_Test::TestBody()’:
../c10/test/util/complex_math_test_common.h:503:1: internal compiler error: Aborted
 }
 ^
0xb3805f crash_signal
        ../../gcc/toplev.c:325
0x7fe84604624f ???
        /usr/src/debug/glibc-2.17-c758a686/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
0x7fe8460461d7 __GI_raise
        ../nptl/sysdeps/unix/sysv/linux/raise.c:56
0x7fe8460478c7 __GI_abort
        /usr/src/debug/glibc-2.17-c758a686/stdlib/abort.c:90
0x8df310 do_mpc_arg1
        ../../gcc/fold-const-call.c:346
0x8df310 do_mpc_arg1
        ../../gcc/fold-const-call.c:326
0x8e0cf2 fold_const_call_cc
        ../../gcc/fold-const-call.c:1032
0x8e0cf2 fold_const_call_1
        ../../gcc/fold-const-call.c:1125
0xe42331 gimple_resimplify1(gimple**, code_helper*, tree_node*, tree_node**, tree_node* (*)(tree_node*))
        ../../gcc/gimple-match-head.c:89
0xf06e05 gimple_simplify(gimple*, code_helper*, tree_node**, gimple**, tree_node* (*)(tree_node*), tree_node* (*)(tree_node*))
        ../../gcc/gimple-match-head.c:782
0x904bca gimple_fold_stmt_to_constant_1(gimple*, tree_node* (*)(tree_node*), tree_node* (*)(tree_node*))
        ../../gcc/gimple-fold.c:6107
0xcadc89 visit_use
        ../../gcc/tree-ssa-sccvn.c:4132
0xcafa46 process_scc
        ../../gcc/tree-ssa-sccvn.c:4346
0xcafa46 extract_and_process_scc_for_name
        ../../gcc/tree-ssa-sccvn.c:4448
0xcafa46 DFS
        ../../gcc/tree-ssa-sccvn.c:4498
0xcafdc7 sccvn_dom_walker::before_dom_children(basic_block_def*)
        ../../gcc/tree-ssa-sccvn.c:4940
0x11f40f7 dom_walker::walk(basic_block_def*)
        ../../gcc/domwalk.c:353
0xcb0a98 run_scc_vn(vn_lookup_kind)
        ../../gcc/tree-ssa-sccvn.c:5047
0xcb126a execute
        ../../gcc/tree-ssa-sccvn.c:6080
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See &lt;https://gcc.gnu.org/bugs/&gt; for instructions.
[2/5] Building CXX object caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o
FAILED: caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o
/home/gakadam/tools/gcc-8.4/bin/g++  -DBUILDING_TESTS -DFMT_HEADER_ONLY=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMAGMA_V2 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTHP_BUILD_MAIN_LIB -DTH_BLAS_MKL -DUSE_C10D -DUSE_C10D_GLOO -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_CUDNN -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_NCCL -DUSE_NUMPY -DUSE_TENSORPIPE -D_FILE_OFFSET_BITS=64 -Dtorch_python_EXPORTS -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -I../torch/.. -I../torch/../aten/src -I../torch/../aten/src/TH -Icaffe2/aten/src -Ithird_party -I../torch/../third_party/gloo -I../torch/../third_party/onnx -I../torch/csrc -I../torch/csrc/api/include -I../torch/lib -I../torch/lib/libshm -I../torch/csrc/api -I../c10/.. -Ithird_party/ideep/mkl-dnn/include -I../third_party/ideep/mkl-dnn/src/../include -I../c10/cuda/../.. -I../torch/lib/libshm/../../../torch/lib -I../third_party/fmt/include -I../torch/lib/c10d/.. -I../torch/lib/c10d/../../.. -Inccl/include -I../third_party/tensorpipe -Ithird_party/tensorpipe -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /home/gakadam/tools/anaconda3/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /home/gakadam/tools/anaconda3/include/python3.8 -isystem /home/gakadam/tools/anaconda3/lib/python3.8/site-packages/numpy/core/include -isystem ../cmake/../third_party/pybind11/include -isystem ../cmake/../third_party/cub -isystem /home/gakadam/tools/cuda-10.2/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -fPIC   -DCAFFE2_USE_GLOO -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -fno-strict-aliasing -Wno-write-strings -Wno-strict-aliasing -pthread -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-unknown-pragmas -std=gnu++14 -MD -MT caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o -MF caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o.d -o caffe2/torch/CMakeFiles/torch_python.dir/csrc/DataLoader.cpp.o -c ../torch/csrc/DataLoader.cpp
../torch/csrc/DataLoader.cpp: In function ‘PyObject* THPModule_removeWorkerPIDs(PyObject*, PyObject*)’:
../torch/csrc/DataLoader.cpp:185:21: error: expected primary-expression before ‘(’ token
     throw ValueError("Cannot find worker information for _BaseDataLoaderIter with id %" PRId64, key);
                     ^
../torch/csrc/DataLoader.cpp:185:88: error: expected ‘)’ before ‘PRId64’
     throw ValueError("Cannot find worker information for _BaseDataLoaderIter with id %" PRId64, key);
                     ~                                                                  ^~~~~~~
                                                                                        )
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 747, in &lt;module&gt;
    build_deps()
  File "setup.py", line 326, in build_deps
    build_caffe2(version=version,
  File "/home/gakadam/tools/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "/home/gakadam/tools/pytorch/tools/setup_helpers/cmake.py", line 345, in build
    self.run(build_args, my_env)
  File "/home/gakadam/tools/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/home/gakadam/tools/anaconda3/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '56']' returned non-zero exit status 1.
</code></pre>
<p>How to rectify this? Please let me know if more details are needed.</p>
<p>thank you.</p>

albanD: <p>Hi,</p>
<p>This is surprising indeed.<br>
Can you try a different gcc version to make sure that the “internal compiler error” is just due to gcc 8.4?<br>
If it is, can you please open an issue on github so that we can fix it <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

gakadam: <p>I get the same result with gcc-7.5.0 / CUDA-10.2.</p>

albanD: <p>Our ci only tests 4.8, 5.4 and 7.2 IIRC.<br>
But this file wasn’t changed for several month so I’m a bit surprise that it would trigger an internal compiler error… Is there anything specific about the way you install gcc? Also which distro are you using?</p>

prashantserai: <p>I’m facing the same error with gcc 6.3.1 /cuda-10.2 on RHEL 6</p>

malfet: <p>And will it pass if you are to build PyTorch without CUDA?<br>
No luck reproducing it with CUDA-10.2 + gcc-8.3.0…</p>

malfet: <p>Can you share a bit more info on how /home/gakadam/tools/gcc-8.4/bin/g++ was built?<br>
I can not reproduce the same crash using gcc-8.4 from Ubuntu-18.04:</p>
<pre><code class="lang-auto">/pytorch/build# g++-8  -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMAGMA_V2 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -I../c10/.. -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /home/gakadam/tools/anaconda3/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /home/gakadam/tools/anaconda3/include/python3.8 -isystem /home/gakadam/tools/anaconda3/lib/python3.8/site-packages/numpy/core/include -isystem ../cmake/../third_party/pybind11/include -isystem ../cmake/../third_party/cub -isystem /home/gakadam/tools/cuda-10.2/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -isystem ../third_party/googletest/googlemock/include -isystem ../third_party/googletest/googlemock -isystem ../third_party/googletest/googletest/include -isystem ../third_party/googletest/googletest -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -fPIE   -DCAFFE2_USE_GLOO -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -pthread -std=gnu++14 -MD -MT c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o -MF c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o.d -o c10/test/CMakeFiles/c10_complex_math_test.dir/util/complex_math_test.cpp.o -c ../c10/test/util/complex_math_test.cpp
root@f02210591fe6:/pytorch/build# g++-8
g++-8: fatal error: no input files
compilation terminated.
root@f02210591fe6:/pytorch/build# g++-8 -v
Using built-in specs.
COLLECT_GCC=g++-8
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 8.4.0-1ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.4.0 (Ubuntu 8.4.0-1ubuntu1~18.04) 
</code></pre>

prashantserai: <p>I just realized that this was just part of the logs and not the terminal error in my case, the terminal error was just below this so I got fooled. When I cleaned my install and tried again, the terminal error was further different and this message was nowhere to be seen near it in the logs.</p>
<p><a class="mention" href="/u/gakadam">@gakadam</a> might still have this issue but I don’t seem to</p>