Issue 1:
Hi,

I'm trying to complile TF from sources I recieve the following error:

dyld: Library not loaded: @rpath/libcudart.8.0.dylib

I know for a fact that cuda is correctly installed as I'm currently using TF on my machine (pip).

I've been following those steps:

clone master

select X-code 7.2

./configure, all defaults except:

compile options: -march=native -mavx -mavx2 -mfma

support cuda [Y]

bazel build --verbose_failures --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

what I'm I doing wrong ?

Thanks

Cesare:tensorflow-master cesare$ bazel build --verbose_failures --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

WARNING: /Users/cesare/Projects/ml/tensorflow-master/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': Use SavedModel Builder instead.
WARNING: /Users/cesare/Projects/ml/tensorflow-master/tensorflow/contrib/learn/BUILD:15:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': Use SavedModel instead.
INFO: Found 1 target...
ERROR: /Users/cesare/Projects/ml/tensorflow-master/tensorflow/contrib/factorization/BUILD:106:1: Executing genrule //tensorflow/contrib/factorization:gen_factorization_ops_pygenrule failed: bash failed: error executing command
(cd /private/var/tmp/_bazel_cesare/bcc912c27d26c81cd2b264ad18985527/execroot/tensorflow-master && \
exec env - \
CUDA_TOOLKIT_PATH=/usr/local/cuda \
CUDNN_INSTALL_PATH=/usr/local/cuda \
GCC_HOST_COMPILER_PATH=/usr/bin/gcc \
PATH=/usr/local/cuda/bin:/opt/local/bin:/opt/local/sbin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin \
TF_CUDA_COMPUTE_CAPABILITIES=3.5,5.2 \
TF_CUDA_VERSION='' \
TF_CUDNN_VERSION='' \
TF_NEED_CUDA=1 \
TMPDIR=/var/folders/n5/2dz3mvjj1cs6cn6j93_vj_mc0000gn/T/ \
/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/contrib/factorization/gen_gen_factorization_ops_py_wrappers_cc 0 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/contrib/factorization/python/ops/gen_factorization_ops.py'): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 6.
dyld: Library not loaded: @rpath/libcudart.8.0.dylib
Referenced from: /private/var/tmp/_bazel_cesare/bcc912c27d26c81cd2b264ad18985527/execroot/tensorflow-master/bazel-out/host/bin/tensorflow/contrib/factorization/gen_gen_factorization_ops_py_wrappers_cc
Reason: image not found
/bin/bash: line 1: 63383 Abort trap: 6 bazel-out/host/bin/tensorflow/contrib/factorization/gen_gen_factorization_ops_py_wrappers_cc 0 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/contrib/factorization/python/ops/gen_factorization_ops.py
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 1.018s, Critical Path: 0.04s

Cesare:tensorflow-master cesare$ ls -la /usr/local/cuda/lib/libcudart.8.0.dylib

lrwxr-xr-x@ 1 cesare staff 50 Sep 27 2016 /usr/local/cuda/lib/libcudart.8.0.dylib -> /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib

Cesare:tensorflow-master cesare$ echo $DYLD_LIBRARY_PATH

/usr/local/cuda/extras/CUPTI/lib:/usr/local/cuda/lib:/usr/local/cuda

Issue 2:
Something related to linking seems to cause the build to fail on Mac OS 10.12.2 when building with CUDA.

Verbose build error is:

$ bazel build -c opt --config=cuda --verbose_failures //tensorflow/tools/pip_package:build_pip_package
INFO: Found 1 target...
ERROR: /Users/anton/tmp/tensorflow/tensorflow/contrib/ffmpeg/BUILD:66:1: Executing genrule //tensorflow/contrib/ffmpeg:decode_audio_op_py_pygenrule failed: bash failed: error executing command
(cd /private/var/tmp/_bazel_anton/d3361bfa15c75c42ab541f3d83e8eba4/execroot/tensorflow && \
exec env - \
PATH=/usr/local/cuda/bin:/Users/anton/.pyenv/shims:/Users/anton/.rbenv/shims:/Users/anton/.scalaenv/shims:/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Users/anton/unix/bin:/usr/local/cuda/bin \
TMPDIR=/var/folders/4r/mq7ht1z11t72w5b5b6014zwm0000gn/T/ \
/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/contrib/ffmpeg/gen_decode_audio_op_py_py_wrappers_cc 1 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/contrib/ffmpeg/ops/gen_decode_audio_op_py.py'): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 6.
dyld: Library not loaded: @rpath/libcudart.8.0.dylib
Referenced from: /private/var/tmp/_bazel_anton/d3361bfa15c75c42ab541f3d83e8eba4/execroot/tensorflow/bazel-out/host/bin/tensorflow/contrib/ffmpeg/gen_decode_audio_op_py_py_wrappers_cc
Reason: image not found
/bin/bash: line 1: 72493 Abort trap: 6 bazel-out/host/bin/tensorflow/contrib/ffmpeg/gen_decode_audio_op_py_py_wrappers_cc 1 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/contrib/ffmpeg/ops/gen_decode_audio_op_py.py
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 2.707s, Critical Path: 0.32s

Related posts

It seems that Caffe had similar issues due to Apple dropping the LD_LIBRARY_PATH environment variable: BVLC/caffe#3227

Environment info

Operating System: Mac OS 10.12.2

Installed version of CUDA and cuDNN:

$ ls -l /usr/local/cuda/lib/libcud*
-rwxr-xr-x 1 root wheel 13504 Nov 3 19:39 /usr/local/cuda/lib/libcuda.dylib*
lrwxr-xr-x 1 root wheel 45 Nov 3 19:40 /usr/local/cuda/lib/libcudadevrt.a@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudadevrt.a
lrwxr-xr-x 1 root wheel 50 Nov 3 19:40 /usr/local/cuda/lib/libcudart.8.0.dylib@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudart.8.0.dylib
lrwxr-xr-x 1 root wheel 46 Nov 3 19:40 /usr/local/cuda/lib/libcudart.dylib@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudart.dylib
lrwxr-xr-x 1 root wheel 49 Nov 3 19:40 /usr/local/cuda/lib/libcudart_static.a@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudart_static.a
lrwxr-xr-x 1 root admin 47 Jan 8 16:48 /usr/local/cuda/lib/libcudnn.5.dylib@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudnn.5.dylib
lrwxr-xr-x 1 root admin 45 Jan 8 16:48 /usr/local/cuda/lib/libcudnn.dylib@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudnn.dylib
lrwxr-xr-x 1 root admin 48 Jan 8 16:48 /usr/local/cuda/lib/libcudnn_static.a@ -> /Developer/NVIDIA/CUDA-8.0/lib/libcudnn_static.a

Configuration command

Please specify the location of python. [Default is /Users/anton/.pyenv/shims/python]:
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] N
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N] N
No Hadoop File System support will be enabled for TensorFlow
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
Found possible Python library paths:
/Users/anton/.pyenv/versions/anaconda3-4.1.1/lib/python3.5/site-packages
Please input the desired Python library path to use. Default is [/Users/anton/.pyenv/versions/anaconda3-4.1.1/lib/python3.5/site-packages]

Using python library path: /Users/anton/.pyenv/versions/anaconda3-4.1.1/lib/python3.5/site-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] Y
CUDA support will be enabled for TensorFlow
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to use system default]:
Please specify the location where CUDA toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify the Cudnn version you want to use. [Leave empty to use system default]:
Please specify the location where cuDNN library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
libcudnn.dylib resolves to libcudnn.dylib
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 3.5
Extracting Bazel installation...
............
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
.........
____Loading package: tensorflow/tools/git
____Loading package: tensorflow/contrib/opt
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 40,960 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 444,518 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 743,716 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 1,170,534 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 1,597,352 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 1,998,646 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 2,251,050 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 2,675,032 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 3,144,390 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 3,579,716 bytes
____Downloading http://bazel-mirror.storage.googleapis.com/github.com/google/protobuf/archive/008b5a228b37c054f46ba478ccafa5e855cb16db.tar.gz: 3,788,162 bytes
INFO: All external dependencies fetched successfully.
Configuration finished

Git revision

$ git rev-parse HEAD
ec7929b878926c39255254e9aea992f0bc65aa68

Bazel version

$ bazel version
Build label: 0.4.3-homebrew
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 22 15:20:15 2016 (1482420015)
Build timestamp: 1482420015
Build timestamp as int: 1482420015

Update

I modified third_party/gpus/crosstool/CROSSTOOL.tpl the following lines

cxx_flag: "-std=c++11"
linker_flag: "-Wl,-no-as-needed"

in the toolchain section to:

cxx_flag: "-std=c++11"
linker_flag: "-Wl,-no-as-needed,-rpath,/usr/local/cuda/lib"

Now I get failures at:

ERROR: /Users/anton/tmp/tensorflow/tensorflow/python/BUILD:793:1: Executing genrule //tensorflow/python:control_flow_ops_pygenrule failed: bash failed: error executing command
(cd /private/var/tmp/_bazel_anton/d3361bfa15c75c42ab541f3d83e8eba4/execroot/tensorflow && \
exec env - \
PATH=/usr/local/cuda/bin:/Users/anton/.pyenv/shims:/Users/anton/.rbenv/shims:/Users/anton/.scalaenv/shims:/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Users/anton/unix/bin:/usr/local/cuda/bin \
TMPDIR=/var/folders/4r/mq7ht1z11t72w5b5b6014zwm0000gn/T/ \
/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/python/gen_control_flow_ops_py_wrappers_cc @tensorflow/python/ops/hidden_ops.txt 1 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/python/ops/gen_control_flow_ops.py'): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 6.
dyld: Library not loaded: @rpath/libcudart.8.0.dylib
Referenced from: /private/var/tmp/_bazel_anton/d3361bfa15c75c42ab541f3d83e8eba4/execroot/tensorflow/bazel-out/host/bin/tensorflow/python/gen_control_flow_ops_py_wrappers_cc
Reason: image not found
/bin/bash: line 1: 45635 Abort trap: 6 bazel-out/host/bin/tensorflow/python/gen_control_flow_ops_py_wrappers_cc @tensorflow/python/ops/hidden_ops.txt 1 > bazel-out/local_darwin-py3-opt/genfiles/tensorflow/python/ops/gen_control_flow_ops.py
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 1683.463s, Critical Path: 1587.37s

I got a bit further but it seems that I need to modify the link options in more files.