Repository: pytorch
Issue: Pytorch compilation error
Labels: module: build,triaged

Hi, 

I tried to build a docker container with the following Dockerfile. But the compilation of pytorch failed with the following error. I succeed building the container with python 2.7. But, when i changed the Dockerfile content to using python 3.5, the error occurs. Does anyone know why this happen and what can i do to solve? Thanks in advance.

```
FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04

RUN apt-get -y update
RUN apt-get install -y --no-install-recommends \
      build-essential \
      git \
      libgoogle-glog-dev \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      libprotobuf-dev \
      openmpi-bin \
      openmpi-doc \
      protobuf-compiler \
      python-dev \
      python3-pip   
RUN pip3 install --upgrade pip
RUN pip3 install setuptools                       
RUN pip3 install --user \
      future \
      nump==1.14.0 \
      protobuf==3.5.1 \
      typing \
      hypothesis \
      mkl \
      mkl-include \
      cffi
RUN apt-get install -y --no-install-recommends \
      libgflags-dev \
      cmake

RUN git clone --branch master --recursive https://github.com/pytorch/pytorch.git
RUN pip3 install typing pyyaml
WORKDIR /pytorch
RUN git submodule update --init --recursive
RUN python3 setup.py install --cmake

RUN git clone https://github.com/facebookresearch/detectron /detectron

# Install Python dependencies
RUN pip3 install -U pip
RUN pip3 install -r /detectron/requirements.txt

# Install the COCO API
RUN git clone https://github.com/cocodataset/cocoapi.git /cocoapi
WORKDIR /cocoapi/PythonAPI

ENV PYTHONPATH /usr/local
ENV Caffe2_DIR=/usr/local/lib/python3.5/dist-packages/torch/share/cmake/Caffe2/
ENV PYTHONPATH=${PYTHONPATH}:/pytorch/build
ENV LD_LIBRARY_PATH=/usr/local/lib:${LD_LIBRARY_PATH}

ENV LD_LIBRARY_PATH=/usr/local/lib/python3.5/dist-packages/torch/lib/:${LD_LIBRARY_PATH}
ENV LIBRARY_PATH=/usr/local/lib/python3.5/dist-packages/torch/lib/:${LIBRARY_PATH}

ENV C_INCLUDE_PATH=/usr/local/lib/python3.5/dist-packages/torch/lib/include/:${C_INCLUDE_PATH}
ENV CPLUS_INCLUDE_PATH=/usr/local/lib/python3.5/dist-packages/torch/lib/include/:${CPLUS_INCLUDE_PATH}

ENV C_INCLUDE_PATH=/pytorch/:${C_INCLUDE_PATH}
ENV CPLUS_INCLUDE_PATH=/pytorch/:${CPLUS_INCLUDE_PATH}

ENV C_INCLUDE_PATH=/pytorch/build/:${C_INCLUDE_PATH}
ENV CPLUS_INCLUDE_PATH=/pytorch/build/:${CPLUS_INCLUDE_PATH}

ENV C_INCLUDE_PATH=/pytorch/torch/lib/include/:${C_INCLUDE_PATH}
ENV CPLUS_INCLUDE_PATH=/pytorch/torch/lib/include/:${CPLUS_INCLUDE_PATH}

RUN make install

WORKDIR /detectron

RUN make
#RUN make ops

RUN apt-get -y update \
    && apt-get -y install \
        wget \
        software-properties-common \
        nano \
        iputils-ping
```

```
[ 85%] Building CXX object caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o
[ 85%] Building CXX object caffe2/CMakeFiles/generate_proposals_op_util_nms_gpu_test.dir/operators/generate_proposals_op_util_nms_gpu_test.cc.o
[ 85%] Built target batch_permutation_op_gpu_test
[ 85%] Building CXX object caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_dlpack.cc.o
In file included from /pytorch/third_party/pybind11/include/pybind11/pytypes.h:12:0,
                 from /pytorch/third_party/pybind11/include/pybind11/cast.h:13,
                 from /pytorch/third_party/pybind11/include/pybind11/attr.h:13,
                 from /pytorch/third_party/pybind11/include/pybind11/pybind11.h:44,
                 from /pytorch/caffe2/python/pybind_state_dlpack.h:9,
                 from /pytorch/caffe2/python/pybind_state.h:16,
                 from /pytorch/caffe2/python/pybind_state.cc:1:
/pytorch/third_party/pybind11/include/pybind11/detail/common.h:112:20: fatal error: Python.h: No such file or directory
compilation terminated.
caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/build.make:62: recipe for target 'caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o' failed
make[2]: *** [caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /pytorch/third_party/pybind11/include/pybind11/pytypes.h:12:0,
                 from /pytorch/third_party/pybind11/include/pybind11/cast.h:13,
                 from /pytorch/third_party/pybind11/include/pybind11/attr.h:13,
                 from /pytorch/third_party/pybind11/include/pybind11/pybind11.h:44,
                 from /pytorch/caffe2/python/pybind_state_dlpack.h:9,
                 from /pytorch/caffe2/python/pybind_state_dlpack.cc:1:
/pytorch/third_party/pybind11/include/pybind11/detail/common.h:112:20: fatal error: Python.h: No such file or directory
compilation terminated.
caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/build.make:86: recipe for target 'caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_dlpack.cc.o' failed
make[2]: *** [caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/python/pybind_state_dlpack.cc.o] Error 1
CMakeFiles/Makefile2:4424: recipe for target 'caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/all' failed
make[1]: *** [caffe2/CMakeFiles/caffe2_pybind11_state_gpu.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Scanning dependencies of target math_gpu_test
[ 85%] Building CXX object caffe2/CMakeFiles/math_gpu_test.dir/utils/math_gpu_test.cc.o
[ 85%] Linking CXX executable ../bin/cuda_cudnn_test
[ 85%] Built target cuda_cudnn_test
[ 85%] Linking CXX executable ../bin/cuda_stream_test
[ 85%] Built target cuda_stream_test
[ 85%] Linking CXX executable ../bin/cuda_apply_test
[ 85%] Built target cuda_apply_test
[ 85%] Linking CXX executable ../bin/blob_gpu_test
[ 85%] Linking CXX executable ../bin/cuda_tensor_interop_test
[ 85%] Built target blob_gpu_test
[ 85%] Built target cuda_tensor_interop_test
[ 85%] Linking CXX executable ../bin/math_gpu_test
[ 85%] Built target math_gpu_test
[ 85%] Linking CXX executable ../bin/generate_proposals_op_util_nms_gpu_test
[ 85%] Built target generate_proposals_op_util_nms_gpu_test
make: *** [all] Error 2
Makefile:138: recipe for target 'all' failed
-- Building version 1.5.0a0+8dc67a0
cmake -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/pytorch/torch -DCMAKE_PREFIX_PATH=/usr/lib/python3/dist-packages -DNUMPY_INCLUDE_DIR=/root/.local/lib/python3.5/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.5m -DPYTHON_LIBRARY=/usr/lib/libpython3.5m.so.1.0 -DTORCH_BUILD_VERSION=1.5.0a0+8dc67a0 -DUSE_NUMPY=True /pytorch
cmake --build . --target install --config Release -- -j 8
Traceback (most recent call last):
  File "setup.py", line 737, in <module>
    build_deps()
  File "setup.py", line 316, in build_deps
    cmake=cmake)
  File "/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "/pytorch/tools/setup_helpers/cmake.py", line 339, in build
    self.run(build_args, my_env)
  File "/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 2
The command '/bin/sh -c python3 setup.py install --cmake' returned a non-zero code: 1
```