Repository: pytorch
Issue: Build problems caffe2 -- pytorch from source & CUDA 11.0
Labels: module: build,module: cuda,triaged

## Issue description

Issue building caffe2 when building Pytorch from source with CUDA 11.0 and cuDNN 8.0

## Code example

```
[4206/4964] Building CXX object caffe2/CMakeF...11_state_gpu.dir/python/pybind_state_gpu.cc.o
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_gpu.cc.o
/usr/bin/c++  -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 
-DIDEEP_USE_MKL -DMAGMA_V2 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DON
NX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 
-Dcaffe2_pybind11_state_gpu_EXPORTS -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_par
ty/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third
_party/foxi -Ithird_party/foxi -Icaffe2/aten/src/TH -I../aten/src/TH -Icaffe2/aten/src -I../at
en/../third_party/catch/single_include -I../aten/src/ATen/.. -Icaffe2/aten/src/ATen -I../third
_party/miniz-2.0.8 -I../caffe2/core/nomnigraph/include -I../torch/csrc/api -I../torch/csrc/api/include -I../c10/.. -Ithird_party/ideep/mkl-dnn/include -I../third_party/ideep/mkl-dnn/src/../include -I../c10/cuda/../.. -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /opt/anaconda3/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /opt/anaconda3/include/python3.6m -isystem /opt/anaconda3/lib/python3.6/site-packages/numpy/core/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include -isystem /usr/local/cuda-11.0/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -fPIC   -fvisibility=hidden -DCAFFE2_USE_GLOO -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -pthread -std=gnu++14 -MD -MT caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_gpu.cc.o -MF caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_gpu.cc.o.d -o caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_gpu.cc.o -c ../caffe2/python/pybind_state_gpu.cc

../caffe2/python/pybind_state_gpu.cc:165:16: error: expected constructor, destructor, or type conversion before ‘(’ token
 PYBIND11_MODULE(caffe2_pybind11_state_gpu, m) {

[4244/4964] Building CXX object caffe2/CMakeF...bind11_state_gpu.dir/python/pybind_state.cc.o
FAILED: caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o
/usr/bin/c++  -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMAGMA_V2 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -Dcaffe2_pybind11_state_gpu_EXPORTS -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -Icaffe2/aten/src/TH -I../aten/src/TH -Icaffe2/aten/src -I../aten/../third_party/catch/single_include -I../aten/src/ATen/.. -Icaffe2/aten/src/ATen -I../third_party/miniz-2.0.8 -I../caffe2/core/nomnigraph/include -I../torch/csrc/api -I../torch/csrc/api/include -I../c10/.. -Ithird_party/ideep/mkl-dnn/include -I../third_party/ideep/mkl-dnn/src/../include -I../c10/cuda/../.. -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /opt/anaconda3/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /opt/anaconda3/include/python3.6m -isystem /opt/anaconda3/lib/python3.6/site-packages/numpy/core/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include -isystem /usr/local/cuda-11.0/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -fPIC   -fvisibility=hidden -DCAFFE2_USE_GLOO -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -pthread -std=gnu++14 -MD -MT caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o -MF caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o.d -o caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o -c ../caffe2/python/pybind_state.cc
../caffe2/python/pybind_state.cc: In function ‘bool caffe2::python::python_detail::feedBlob(caffe2::Blob*, const pybind11::object&, pybind11::object)’:
../caffe2/python/pybind_state.cc:233:22: error: ‘const class pybind11::object’ has no member named ‘is’

   if (!device_option.is(py::none())) {
                      ^~
../caffe2/python/pybind_state.cc: In function ‘void caffe2::python::addObjectMethods(pybind11::module&)’:
../caffe2/python/pybind_state.cc:757:8: error: no matching function for call to ‘init(caffe2::python::addObjectMethods(pybind11::module&)::<lambda(const std::vector<pybind11::bytes>&, const std::vector<pybind11::bytes>&, const std::vector<std::__cxx11::basic_string<char> >&)>)’
       }));
        ^
In file included from ../caffe2/python/pybind_state_dlpack.h:9:0,
                 from ../caffe2/python/pybind_state.h:16,
                 from ../caffe2/python/pybind_state.cc:1:
/usr/include/pybind11/pybind11.h:1418:51: note: candidate: template<class ... Args> pybind11::detail::init<Args ...> pybind11::init()
 template <typename... Args> detail::init<Args...> init() { return detail::init<Args...>(); }
                                                   ^~~~
/usr/include/pybind11/pybind11.h:1418:51: note:   template argument deduction/substitution failed:
../caffe2/python/pybind_state.cc:757:8: note:   candidate expects 0 arguments, 1 provided
       }));
        ^
../caffe2/python/pybind_state.cc: In lambda function:
../caffe2/python/pybind_state.cc:764:22: error: ‘const class pybind11::object’ has no member named ‘is’
             if (args.is(py::none())) {
                      ^~
../caffe2/python/pybind_state.cc: In function ‘void caffe2::python::addObjectMethods(pybind11::module&)’:
../caffe2/python/pybind_state.cc:958:8: error: no matching function for call to ‘init(caffe2::python::addObjectMethods(pybind11::module&)::<lambda(pybind11::bytes, pybind11::bytes)>)’
       }))
        ^
In file included from ../caffe2/python/pybind_state_dlpack.h:9:0,
                 from ../caffe2/python/pybind_state.h:16,
                 from ../caffe2/python/pybind_state.cc:1:
/usr/include/pybind11/pybind11.h:1418:51: note: candidate: template<class ... Args> pybind11::detail::init<Args ...> pybind11::init()

 template <typename... Args> detail::init<Args...> init() { return detail::init<Args...>(); }
                                                   ^~~~
/usr/include/pybind11/pybind11.h:1418:51: note:   template argument deduction/substitution failed:
../caffe2/python/pybind_state.cc:958:8: note:   candidate expects 0 arguments, 1 provided
       }))
        ^
../caffe2/python/pybind_state.cc: In lambda function:
../caffe2/python/pybind_state.cc:1127:31: error: ‘const class pybind11::object’ has no member named ‘is’
         if (create_if_missing.is(py::none())) {
                               ^~
../caffe2/python/pybind_state.cc: In lambda function:
../caffe2/python/pybind_state.cc:1139:25: error: ‘const class pybind11::object’ has no member named ‘is’
         if (root_folder.is(py::none())) {
                         ^~
In file included from ../c10/util/typeid.h:20:0,
                 from ../caffe2/core/context.h:9,
                 from ../caffe2/python/pybind_state.h:5,
                 from ../caffe2/python/pybind_state.cc:1:
../caffe2/python/pybind_state.cc: In lambda function:
../caffe2/python/pybind_state.cc:1598:29: error: ‘class pybind11::object’ has no member named ‘is’
         CAFFE_ENFORCE(!func.is(py::none()));
                             ^
../c10/macros/Macros.h:171:65: note: in definition of macro ‘C10_UNLIKELY’
 #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                 ^~~~
../caffe2/python/pybind_state.cc:1598:9: note: in expansion of macro ‘CAFFE_ENFORCE’
         CAFFE_ENFORCE(!func.is(py::none()));
         ^
../caffe2/python/pybind_state.cc: In lambda function:
../caffe2/python/pybind_state.cc:1614:29: error: ‘class pybind11::object’ has no member named ‘is’
         CAFFE_ENFORCE(!func.is(py::none()));
                             ^
../c10/macros/Macros.h:171:65: note: in definition of macro ‘C10_UNLIKELY’
 #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                 ^~~~
../caffe2/python/pybind_state.cc:1614:9: note: in expansion of macro ‘CAFFE_ENFORCE’
         CAFFE_ENFORCE(!func.is(py::none()));
../c10/macros/Macros.h:171:65: note: in definition of macro ‘C10_UNLIKELY’
 #define C10_UNLIKELY(expr)  (__builtin_expect(static_cast<bool>(expr), 0))
                                                                 ^~~~
../caffe2/python/pybind_state.cc:1614:9: note: in expansion of macro ‘CAFFE_ENFORCE’
         CAFFE_ENFORCE(!func.is(py::none()));
         ^
../caffe2/python/pybind_state.cc: At global scope:
../caffe2/python/pybind_state.cc:1903:16: error: expected constructor, destructor, or type conversion before ‘(’ token
 PYBIND11_MODULE(caffe2_pybind11_state, m) {
                ^
[4245/4964] Building CXX object caffe2/CMakeFiles/blob_test.dir/core/blob_test.cc.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 737, in <module>
    build_deps()
  File "setup.py", line 321, in build_deps
    cmake=cmake)
  File "/opt/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env) 
  File "/opt/pytorch/tools/setup_helpers/cmake.py", line 345, in build
    self.run(build_args, my_env)
  File "/opt/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/opt/anaconda3/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '40']' returned non-zero exit status 1.
```


## System Info
PyTorch version: 1.3.1
Is debug build: No
CUDA used to build PyTorch: Could not collect

OS: Ubuntu 18.04.5 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.14.0

Python version: 3.6 (64-bit runtime)
Is CUDA available: No
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Tesla V100-PCIE-16GB
Nvidia driver version: 450.51.06
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.2
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_train.so.8

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.3.1
[pip3] torchvision==0.4.2
[conda] _pytorch_select           0.1                       cpu_0  
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] magma-cuda110             2.5.2                         1    pytorch
[conda] mkl                       2020.1                      217  
[conda] mkl-include               2020.1                      217  
[conda] mkl-service               2.3.0            py36he904b0f_0  
[conda] mkl_fft                   1.1.0            py36h23d657b_0  
[conda] mkl_random                1.1.1            py36h0573a6f_0  
[conda] numpy                     1.19.1           py36hbc911f0_0  
[conda] numpy-base                1.19.1           py36hfa32c7d_0  
[conda] pytorch                   1.3.1           cpu_py36h62f834f_0  
[conda] torchvision               0.4.2           cpu_py36h9ec355b_0

nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

cc @malfet @ngimel