Issue 1:
Hi,

I am trying to build latest TF with CUDNN 6.0 + CUDA 8.0 on Ubuntu 14 but it failed with the following error message. I found some similar issue (#469) reported in the past, not sure if the latest tip has fixed it?

Any suggestion would be appreciated.

Thanks

Build Command

build command for CUDA that failed

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

build for CPU works well

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

System Info

bazel version : Build label: 0.5.4

CUDA: 8.0

CUDNN 6.0

TF origin/master latest sync as 10/26/17 (cb7cb40 Merge pull request #13972 from taehoonlee/fix_typos)

Error message:

ERROR: $PROJECT_ROOT/tensorflow/tensorflow/stream_executor/BUILD:52:1: undeclared inclusion(s) in rule '//tensorflow/stream_executor:cuda_platform': this rule is missing dependency declarations for the following files included by 'tensorflow/stream_executor/cuda/cuda_blas.cc': '/usr/local/cuda/include/cublas_api.h' '/usr/local/cuda/include/driver_types.h' '/usr/local/cuda/include/host_defines.h' '/usr/local/cuda/include/cuComplex.h' '/usr/local/cuda/include/vector_types.h' '/usr/local/cuda/include/builtin_types.h' '/usr/local/cuda/include/device_types.h' '/usr/local/cuda/include/surface_types.h' '/usr/local/cuda/include/texture_types.h' '/usr/local/cuda/include/cuda_fp16.h' '/usr/local/cuda/include/library_types.h' tensorflow/stream_executor/cuda/cuda_blas.cc: In function 'cudaDataType_t perftools::gputools::cuda::{anonymous}::CUDAComputationType(perftools::gputools::blas::ComputationType)': tensorflow/stream_executor/cuda/cuda_blas.cc:527:1: warning: control reaches end of non-void function [-Wreturn-type] } ^

Issue 2:
Hey guys,

I can compile the cpu version of tensorflow 1.2.0-rc2 without any problem, however, i am blocked when i try to compile the gpu version. I spend almost the whole day to install tensorflow 1.2.0-rc2 gpu version in our cluster, however there is no lucky.

Here is the error from the terminal:

ERROR: /home/hpc-xin/.cache/bazel/_bazel_hpc-xin/c5e302e32d7acb9c3e4fce8142d1cd66/external/nccl_archive/BUILD:33:1: undeclared inclusion(s) in rule '@nccl_archive//:nccl':

this rule is missing dependency declarations for the following files included by 'external/nccl_archive/src/libwrap.cu.cc':

'/gpfs/gpfs1/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include-fixed/limits.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include-fixed/syslimits.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include/stddef.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/new'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/x86_64-unknown-linux-gnu/bits/c++config.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/x86_64-unknown-linux-gnu/bits/os_defines.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/x86_64-unknown-linux-gnu/bits/cpu_defines.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/exception'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/bits/atomic_lockfree_defines.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/bits/exception_ptr.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/bits/exception_defines.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/bits/nested_exception.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include/stdarg.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/cmath'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/bits/cpp_type_traits.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/ext/type_traits.h'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/cstdlib'

'/gpfs/gpfs1/apps2/gcc/5.4.0/include/c++/5.4.0/cstdio'.

Target //tensorflow/tools/pip_package:build_pip_package failed to build

Use --verbose_failures to see the command lines of failed build steps.

INFO: Elapsed time: 22.226s, Critical Path: 5.20s

OS Information

Red Hat Enterprise Linux Workstation release 6.7 (Santiago)

Modules I loaded

gcc/5.4.0, cuda/8.0.61, cudnn/6.0, java/1.8.0_31, sqlite/3.18.0, tcl/8.6.6.8606, python/3.6.1

Here is how i make the configuration

gcc (v5.4.0): /apps2/gcc/5.4.0 (customized path）

enabled cuda (v8.0): /apps2/cuda/8.0.61 (customized path）

enabled cudnn: /apps2/cudnn/6.0 (customized path）

python (v3.6.1, which is compiled with gcc 5.4.0): /apps2/python/3.6.1 (customized path)

Other options are disabled (such as hadoop, google cloud).

Here are some files i modified manually before compiling

(1) /tensorflow-1.2.0-rc2/third_party/gpus/crosstool/CROSSTOOL_clang.tpl

/tensorflow-1.2.0-rc2/third_party/gpus/crosstool/CROSSTOOL_nvcc.tpl

Add the following contents in all of "toolchain":

cxx_builtin_include_directory: "/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include"

cxx_builtin_include_directory: "/apps2/gcc/5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/include-fixed"

cxx_builtin_include_directory: "/apps2/gcc/5.4.0/include/c++/5.4.0"

cxx_builtin_include_directory: "/apps2/gcc/5.4.0/include"

cxx_builtin_include_directory: "/apps2/cuda/8.0.61/include"

cxx_builtin_include_directory: "/apps2/cudnn/6.0/include"

(2) /home/hpc-xin/.cache/bazel/_bazel_hpc-xin/c5e302e32d7acb9c3e4fce8142d1cd66/external/protobuf/protobuf.bzl

Add the following content in "ctx.action"

env=ctx.configuration.default_shell_env

(3) /home/hpc-xin/.cache/bazel/_bazel_hpc-xin/c5e302e32d7acb9c3e4fce8142d1cd66/external/nccl_archive/Makefile

Some modification as below:

CUDA_HOME ?= /usr/local/cuda --> CUDA_HOME ?= /apps2/cuda/8.0.61

I noticed that some of people said this issue could be bypassed by delete the nccl dependence in "/tensorflow-1.2.0-rc2/tensorflow/contrib/BUILD". However, i do not think this is the correct way to solve this issue. I wanna keep this dependence.

Thanks so much for your help. Any comments are appreciated.

Best Regards

Xin