Issue 1:
🐛 Bug

Build 1.0 from source (tried today) has failed on CPU-only macOS. Looks like a linker error, but perhaps something else. Please help!

To Reproduce

Steps to reproduce the behavior:

Three commands on command line.

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
NO_CUDA=1 USE_CUDA=0 CC=clang CXX=clang++ python3 setup.py install

Stack trace for build (output in full below in Additional context

[snipped]
[ 40%] Built target onnx
[ 40%] Linking CXX shared library ../lib/libcaffe2.dylib
ld: warning: directory not found for option '-L/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib'
duplicate symbol __ZN2at6native13pixel_shuffleERKNS_6TensorEx in:
CMakeFiles/caffe2.dir/__/aten/src/ATen/native/PixelShuffle_2.cpp.o
CMakeFiles/caffe2.dir/__/aten/src/ATen/native/PixelShuffle.cpp.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libcaffe2.dylib] Error 1
make[1]: *** [caffe2/CMakeFiles/caffe2.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 40%] Built target python_copy_files
make: *** [all] Error 2
Failed to run 'bash ../tools/build_pytorch_libs.sh --use-nnpack caffe2 libshm THD'

Expected behavior

PyTorch build from source such that we can import torch.

Environment

Please copy and paste the output from our

[environment collection script - does not run because torch is not installed]

PyTorch Version (e.g., 1.0): PyTorch source version (1.0 currently)

OS (e.g., Linux): macOS High Sierra

How you installed PyTorch (conda, pip, source): source

Build command you used (if compiling from source):

NO_CUDA=1 USE_CUDA=0 CC=clang CXX=clang++ python3 setup.py install

Python version:

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

CUDA/cuDNN version: N/A (not building with CUDA, or this is the goal as system is CPU-only)

GPU models and configuration: N/A

Any other relevant information: The system does have CUDA 10.0 and cuDNN 7.3.0, but this is not a target.

Additional context

Entire output from build:

umwhat:pytorch micheleenharris$ NO_CUDA=1 USE_CUDA=0 CC=clang CXX=clang++ python3 setup.py install
Building wheel torch-1.0.0a0+9c49bb9
running install
setup.py::run()
running build_deps
setup.py::build_deps::run()
+ SYNC_COMMAND=cp
++ command -v rsync
+ '[' -x /usr/bin/rsync ']'
+ SYNC_COMMAND='rsync -lptgoD'
+ USE_CUDA=0
+ USE_ROCM=0
+ USE_NNPACK=0
+ USE_MKLDNN=0
+ USE_GLOO_IBVERBS=0
+ CAFFE2_STATIC_LINK_CUDA=0
+ RERUN_CMAKE=1
+ [[ 4 -gt 0 ]]
+ case "$1" in
+ USE_NNPACK=1
+ shift
+ [[ 3 -gt 0 ]]
+ case "$1" in
+ break
+ CMAKE_INSTALL='make install'
+ BUILD_SHARED_LIBS=ON
+ USER_CFLAGS=
+ USER_LDFLAGS=
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
++ uname
+ '[' Darwin == Darwin ']'
+ '[' -d /usr/local/opt/ccache/libexec ']'
+++ dirname ../tools/build_pytorch_libs.sh
++ cd ../tools/..
+++ pwd
++ printf '%q\n' /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch
+ BASE_DIR=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch
+ TORCH_LIB_DIR=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib
+ INSTALL_DIR=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install
+ THIRD_PARTY_DIR=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/third_party
+ CMAKE_VERSION=cmake
+ C_FLAGS=' -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/TH" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THC" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THNN" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCUNN"'
+ C_FLAGS=' -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/TH" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THC" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THNN" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1'
+ LDFLAGS='-L"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib" '
+ LD_POSTFIX=.so
++ uname
+ [[ Darwin == \D\a\r\w\i\n ]]
+ LDFLAGS='-L"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,@loader_path'
+ LD_POSTFIX=.dylib
+ CPP_FLAGS=' -std=c++11 '
+ GLOO_FLAGS='-DBUILD_TEST=OFF '
+ THD_FLAGS=
+ NCCL_ROOT_DIR=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install
+ [[ 0 -eq 1 ]]
+ [[ 0 -eq 1 ]]
+ CWRAP_FILES='/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/ATen/Declarations.cwrap;/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/THNN/generic/THNN.h;/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/THCUNN/generic/THCUNN.h;/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/ATen/nn.yaml'
+ CUDA_NVCC_FLAGS=' -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/TH" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THC" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCS" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THNN" -I"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1'
+ [[ -z '' ]]
+ CUDA_DEVICE_DEBUG=0
+ '[' -z '' ']'
++ getconf _NPROCESSORS_ONLN
+ MAX_JOBS=8
+ BUILD_TYPE=Release
+ [[ -n '' ]]
+ [[ -n '' ]]
+ echo 'Building in Release mode'
Building in Release mode
+ mkdir -p /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install
+ for arg in '"$@"'
+ [[ caffe2 == \n\c\c\l ]]
+ [[ caffe2 == \g\l\o\o ]]
+ [[ caffe2 == \c\a\f\f\e\2 ]]
+ build_caffe2
+ [[ -z '' ]]
+ EXTRA_CAFFE2_CMAKE_FLAGS=()
+ [[ -n '' ]]
+ [[ -n /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages ]]
+ EXTRA_CAFFE2_CMAKE_FLAGS+=("-DCMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH")
+ [[ 1 -eq 1 ]]
+ cmake /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch -DCMAKE_INSTALL_MESSAGE=LAZY -DPYTHON_EXECUTABLE=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3 -DBUILDING_WITH_TORCH_LIBS=ON -DTORCH_BUILD_VERSION=1.0.0a0+9c49bb9 -DCMAKE_BUILD_TYPE=Release -DBUILD_TORCH=ON -DBUILD_PYTHON=ON -DBUILD_SHARED_LIBS=ON -DBUILD_BINARY=OFF -DBUILD_TEST=ON -DINSTALL_TEST=ON -DBUILD_CAFFE2_OPS=ON -DONNX_NAMESPACE=onnx_torch -DUSE_CUDA=0 -DCAFFE2_STATIC_LINK_CUDA=0 -DUSE_ROCM=0 -DUSE_NNPACK=1 -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DUSE_OPENCV=OFF -DUSE_GLOG=OFF -DUSE_GFLAGS=OFF -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DCUDNN_INCLUDE_DIR= -DCUDNN_LIB_DIR=//Developer/NVIDIA/CUDA-10.0/lib -DCUDNN_LIBRARY= -DUSE_MKLDNN=0 -DMKLDNN_INCLUDE_DIR= -DMKLDNN_LIB_DIR= -DMKLDNN_LIBRARY= -DCMAKE_INSTALL_PREFIX=/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= '-DCMAKE_EXE_LINKER_FLAGS=-L"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,@loader_path ' '-DCMAKE_SHARED_LINKER_FLAGS=-L"/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,@loader_path ' -DCMAKE_PREFIX_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
-- CLANG_VERSION_STRING: 9.1
-- Need to define long as a separate typeid.
-- std::exception_ptr is supported.
-- NUMA is disabled
-- Turning off deprecation warning due to glog.
-- Current compiler supports avx2 extention. Will build perfkernels.
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- The BLAS backend of choice:MKL
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl - guide - pthread - m]
-- Library mkl: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl - guide - pthread - m]
-- Library mkl: not found
CMake Warning at cmake/Dependencies.cmake:74 (message):
MKL could not be found. Defaulting to Eigen
Call Stack (most recent call first):
CMakeLists.txt:179 (include)

-- Brace yourself, we are building NNPACK
-- Found PythonInterp: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 (found version "3.6.6")
-- LLVM FileCheck Found: /usr/local/opt/llvm/bin/FileCheck
-- git Version: v1.4.0-505be96a
-- Version: 1.4.0
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES -- failed to compile
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Using third party subdirectory Eigen.
Python 3.6.6
-- Setting Python's include dir to /Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m from distutils.sysconfig
-- Setting Python's library to /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6
-- Found PythonInterp: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 (found suitable version "3.6.6", minimum required is "2.7")
-- NumPy ver. 1.15.2 found (include: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include)
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
-- Using third_party/pybind11.
--
-- ******** Summary ********
-- CMake version : 3.9.0-rc5
-- CMake command : /usr/local/bin/cmake
-- System : Darwin
-- C++ compiler : /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- C++ compiler version : 9.1.0.9020039
-- CXX flags : -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations
-- Build type : Release
-- Compile definitions :
-- CMAKE_PREFIX_PATH : /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
-- CMAKE_INSTALL_PREFIX : /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install
-- CMAKE_MODULE_PATH : /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/cmake/Modules
--
-- ONNX version : 1.3.0
-- ONNX NAMESPACE : onnx_torch
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_BENCHMARKS : OFF
-- ONNX_USE_LITE_PROTO : OFF
-- ONNXIFI_DUMMY_BACKEND : OFF
--
-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFF
-- Could not find CUDA with FP16 support, compiling without torch.CudaHalfTensor
-- Removing -DNDEBUG from compile flags
-- MAGMA not found. Compiling without MAGMA support
-- Could not find hardware support for NEON on this machine.
-- No OMAP3 processor on this machine.
-- No OMAP4 processor on this machine.
-- SSE2 Found
-- SSE3 Found
-- AVX Found
-- AVX2 Found
-- Atomics: using GCC intrinsics
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_sequential - mkl_core - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - guide - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel_lp64: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
-- Library mkl_intel: not found
-- Checking for [mkl - guide - pthread - m]
-- Library mkl: not found
-- MKL library not found
-- Checking for [Accelerate]
-- Library Accelerate: /System/Library/Frameworks/Accelerate.framework
-- This BLAS uses the F2C return conventions
-- Found a library with BLAS API (accelerate).
-- Found a library with LAPACK API. (accelerate)
disabling CUDA because NOT USE_CUDA is set
-- CuDNN not found. Compiling without CuDNN support
disabling ROCM because NOT USE_ROCM is set
-- MIOpen not found. Compiling without MIOpen support
-- Could NOT find MKLDNN (missing: MKLDNN_INCLUDE_DIR MKLDNN_LIBRARY)
-- MKLDNN not found. Compiling without MKLDNN support
-- Using python found in /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
disabling CUDA because USE_CUDA is set false
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) (found version "1.0")
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) (found version "1.0")
-- Configuring build for SLEEF-v3.2
Target system: Darwin-17.7.0
Target processor: x86_64
Host system: Darwin-17.7.0
Host processor: x86_64
Detected C compiler: AppleClang @ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Using option `-Wall -Wno-unused -Wno-attributes -Wno-unused-result -ffp-contract=off -fno-math-errno -fno-trapping-math` to compile libsleef
-- Building shared libs : OFF
-- MPFR : /usr/local/lib/libmpfr.dylib
-- MPFR header file in /usr/local/include
-- GMP : /usr/local/lib/libgmp.dylib
-- RUNNING_ON_TRAVIS : 0
-- COMPILER_SUPPORTS_OPENMP :
-- Using python found in /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) (found version "1.0")
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) (found version "1.0")
-- NCCL operators skipped due to no CUDA support
-- Excluding ideep operators as we are not using ideep
-- Excluding image processing operators due to no opencv
-- Excluding video processing operators due to no opencv
-- Excluding mkl operators as we are not using mkl
-- MPI operators skipped due to no MPI support
-- Include Observer library
-- Using lib/python3.6/site-packages as python relative installation path
-- Automatically generating missing __init__.py files.
-- A previous caffe2 cmake run already created the __init__.py files.
CMake Warning at CMakeLists.txt:363 (message):
Generated cmake files are only fully tested if one builds with system glog,
gflags, and protobuf. Other settings may generate files that are not well
tested.

--
-- ******** Summary ********
-- General:
-- CMake version : 3.9.0-rc5
-- CMake command : /usr/local/bin/cmake
-- System : Darwin
-- C++ compiler : /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- C++ compiler version : 9.1.0.9020039
-- BLAS : MKL
-- CXX flags : -msse4.2 --std=c++11 -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -faligned-new -Wno-unused-private-field -Wno-missing-braces -Wno-c++14-extensions -Wno-constexpr-not-const
-- Build type : Release
-- Compile definitions : ONNX_NAMESPACE=onnx_torch;USE_GCC_ATOMICS=1;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1
-- CMAKE_PREFIX_PATH : /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
-- CMAKE_INSTALL_PREFIX : /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install
--
-- TORCH_VERSION : 1.0.0
-- CAFFE2_VERSION : 1.0.0
-- BUILD_ATEN_MOBILE : OFF
-- BUILD_BINARY : OFF
-- BUILD_CUSTOM_PROTOBUF : ON
-- Link local protobuf : ON
-- BUILD_DOCS : OFF
-- BUILD_PYTHON : ON
-- Python version : 3.6.6
-- Python executable : /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
-- Pythonlibs version : 3.6.6
-- Python library : /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6
-- Python includes : /Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m
-- Python site-packages: lib/python3.6/site-packages
-- BUILD_CAFFE2_OPS : ON
-- BUILD_SHARED_LIBS : ON
-- BUILD_TEST : ON
-- USE_ASAN : OFF
-- USE_CUDA : 0
-- USE_ROCM : OFF
-- USE_EIGEN_FOR_BLAS : ON
-- USE_FFMPEG : OFF
-- USE_GFLAGS : OFF
-- USE_GLOG : OFF
-- USE_LEVELDB : OFF
-- USE_LITE_PROTO : OFF
-- USE_LMDB : OFF
-- USE_METAL : OFF
-- USE_MKL :
-- USE_MOBILE_OPENGL : OFF
-- USE_NCCL : OFF
-- USE_NERVANA_GPU : OFF
-- USE_NNPACK : 1
-- USE_OBSERVERS : ON
-- USE_OPENCL : OFF
-- USE_OPENCV : OFF
-- USE_OPENMP : OFF
-- USE_PROF : OFF
-- USE_REDIS : OFF
-- USE_ROCKSDB : OFF
-- USE_ZMQ : OFF
-- USE_DISTRIBUTED : ON
-- USE_MPI : OFF
-- USE_GLOO : OFF
-- USE_GLOO_IBVERBS : OFF
-- Public Dependencies : Threads::Threads
-- Private Dependencies : nnpack;cpuinfo;aten_op_header_gen;onnxifi_loader
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:

CUDNN_LIB_DIR

-- Build files have been written to: /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/build
+ '[' -f /Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib/libnccl.so ']'
+ make install -j8
[ 0%] Built target js_embed
[ 0%] Built target pthreadpool
[ 0%] Built target onnxifi_loader
[ 1%] Built target gtest
[ 1%] Built target clog
[ 3%] Built target benchmark
[ 5%] Built target libprotobuf-lite
[ 5%] Built target onnxifi_dummy
[ 5%] Built target mkrename
[ 5%] Built target common
[ 6%] Built target c10
[ 7%] Built target mkdisp
[ 7%] Built target mkalias
[ 7%] Built target ATEN_CPU_FILES_GEN_TARGET
[ 7%] Built target mkmasked_gnuabi
[ 7%] Built target arraymap
[ 7%] Built target mkrename_gnuabi
[ 11%] Built target c10_utils_hip
[ 13%] Built target libprotobuf
[ 13%] Built target c10_utils_gpu
[ 13%] Built target c10_utils_cpu
[ 14%] Built target nnpack_reference_layers
[ 14%] Built target benchmark_main
[ 14%] Built target gtest_main
[ 15%] Built target onnxifi_wrapper
[ 17%] Built target cpuinfo
[ 18%] Built target renamedsp256.h_generated
[ 18%] Built target dispavx.c_generated
[ 18%] Built target renameSSE2.h_generated
[ 18%] Built target renameAVX.h_generated
[ 18%] Built target renameFMA4.h_generated
[ 19%] Built target headers
[ 19%] Built target renameSSE4.h_generated
[ 19%] Built target renameAVX2128.h_generated
[ 19%] Built target renameAVX2.h_generated
[ 19%] Built target dispsse.c_generated
[ 19%] Built target renamedsp128.h_generated
[ 19%] Built target renameAVX512F.h_generated
[ 19%] Generating alias_avx512f.h
[ 20%] Built target __aten_op_header_gen
[ 20%] Built target c10_utils_gpu_test
[ 20%] Built target c10_utils_hip_test
[ 21%] Built target c10_utils_cpu_test
[ 21%] Built target alias_avx512f.h_generated
[ 23%] Built target nnpack
[ 23%] Built target c10_registry_test
[ 23%] Built target dispavx_obj
[ 23%] Built target sleefsse2
[ 24%] Built target sleefavx
[ 25%] Built target sleeffma4
[ 25%] Built target sleefsse4
[ 25%] Built target sleefavx2128
[ 25%] Built target sleefavx2
[ 25%] Built target dispsse_obj
[ 25%] Built target sleefavx512f
[ 25%] Built target sleef
[ 33%] Built target libprotoc
[ 33%] Built target protoc
[ 33%] Built target gen_onnx_proto
[ 34%] Built target Caffe2_PROTO
[ 34%] Built target onnx_proto
[ 35%] Built target caffe2_protos
[ 35%] Built target Caffe2_perfkernels_avx
[ 36%] Built target dispatch
[ 37%] Built target Caffe2_perfkernels_avx2
[ 37%] Built target dispatch_test
[ 40%] Built target onnx
[ 40%] Linking CXX shared library ../lib/libcaffe2.dylib
ld: warning: directory not found for option '-L/Users/micheleenharris/Documents/bin/github/pytorch-mask-rcnn-samples/pytorch/torch/lib/tmp_install/lib'
duplicate symbol __ZN2at6native13pixel_shuffleERKNS_6TensorEx in:
CMakeFiles/caffe2.dir/__/aten/src/ATen/native/PixelShuffle_2.cpp.o
CMakeFiles/caffe2.dir/__/aten/src/ATen/native/PixelShuffle.cpp.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libcaffe2.dylib] Error 1
make[1]: *** [caffe2/CMakeFiles/caffe2.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 40%] Built target python_copy_files
make: *** [all] Error 2
Failed to run 'bash ../tools/build_pytorch_libs.sh --use-nnpack caffe2 libshm THD'

Issue 2:
For advanced indexing with the duplicate indexes, it is expected that the last one should win, e.g. with the input

0 1 2 3 4
5 6 7 8 9
10 11 12 13 14
15 16 17 18 19

indexer = [slice(None, None, None), [0, 1, 1, 2, 2]] and val

16 12 0 14 10
19 17 2 7 4
5 9 6 1 15
13 11 3 8 18

the expected output of input[indexer]=val is

16 0 10 3 4
19 2 4 8 9
5 6 15 13 14
13 3 18 18 19

there are tests checking that. Yet the kernels doing the indexing (indexCopyLargeIndex and indexCopySmallIndex) provide no such guarantee - if the same dstOffset is shared by several threads, there is no guarantee on which write would be last, and no effort is made before launching the kernels to make sure that dstOffsets would be unique for all threads. This is not really fixable until the indices are de-duplicated and only the unique dstOffsets are present in those kernels. I don't know the best way to handle this. Disable advanced indexing with dupes, say results is undefined, de-dupe the indices? @killeent