Issue 1:
Please go to Stack Overflow for help and support:

https://stackoverflow.com/questions/tagged/tensorflow

If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).

The form below must be filled out.

It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):

NO.

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Slackware Linux 14.2 64 bit

TensorFlow installed from (source or binary):

source

TensorFlow version (use command below):

1.5.0

Python version:

3.6.4

Bazel version (if compiling from source):

0.5.4

GCC/Compiler version (if compiling from source):

5.5.0

CUDA/cuDNN version:

9.0/7

GPU model and memory:

1050Ti / 4Gb

Exact command to reproduce:

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.

gcc 5.5.0 does not compile TF source code.

First of all, sorry for that I'm NOT using Ubuntu Linux for TF. I know that Ubuntu Linux is the only supported Linux system for TF. Instead, I use an ancient distribution, i.e., Slackware Linux. Recently, I got a security update for Spectre. As a side-effect, I also got an updated gcc (5.3.0 -> 5.5.0).

I usually compile the TensorFlow source code for optimization. However, it can not be compiled with updated gcc (5.5.0), like this;

INFO: From Compiling tensorflow/contrib/resampler/kernels/resampler_ops_gpu.cu.cc:
/usr/lib64/gcc/x86_64-slackware-linux/5.5.0/include/avx512fintrin.h(9220): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib64/gcc/x86_64-slackware-linux/5.5.0/include/avx512fintrin.h(9231): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib64/gcc/x86_64-slackware-linux/5.5.0/include/avx512fintrin.h(9244): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib64/gcc/x86_64-slackware-linux/5.5.0/include/avx512fintrin.h(9255): error: argument of type "const void *" is incompatible with parameter of type "const double *"

So I googled a little bit, and found the following issue:

#10220

in the middle of the thread, I saw

I think the problem here is that gcc-5.5 shipped with avx512intrin.h headers that switched to using void and const void* (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=76731) but without switching the builtins to do the same. This is why 5.4 works but 5.5 breaks.

so I tracked down the matter that I could see

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=76731

All of the scatter/gather intrinsics in avx512intrin.h use int/float/double pointers, which is incorrect.

So, at first, I thought this problem is about gcc, but someone suggested that it's maybe related to CUDA.

it's the GPU/CUDA code that doesn't support the new compiler so, if you want to build that, your only option is downgrade the compiler, until Nvidia releases a new CUDA sdk.

from https://www.linuxquestions.org/questions/showthread.php?p=5817669#post5817669

To wrap up, I was able to compile TF successfully with gcc 5.3.0. With gcc 5.5.0, I get error messages here and there, and yet I do not know what makes this errors. I suspect the combination of gcc and CUDA (and also TensorFlow) does not work well, but I still can not figure out which of them makes this fault.

Thank you for your help.

Best regards,

sungjin.

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04

TensorFlow installed from (source or binary): building from source

TensorFlow version (use command below): 5ae244e

Bazel version (if compiling from source): 0.4.5

CUDA/cuDNN version: CUDA 8.0.61, cudnn 6.0.21 (tried also 5.1)

GPU model and memory: 2x Tesla P100-PCIE-12GB

Exact command to reproduce: building

Additional information: Intel(R) Xeon(R) CPU E7-4860 v2 @ 2.60GHz, gcc version 5.4.1 20170519 (Ubuntu 5.4.1-11ubuntu2~16.04)

Describe the problem

On the regular rebuild of Tensorflow, the build crashes with bunch of error: argument of type "const void *" is incompatible with parameter of type "const something *"

Source code / logs

Crash log:

INFO: From Compiling tensorflow/core/kernels/scatter_functor_gpu.cu.cc:
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9218): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9229): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9242): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9253): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9266): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9277): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9290): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9301): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9314): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9325): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9338): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9350): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9363): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9374): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9387): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9399): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9408): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9417): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9426): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9435): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9443): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9452): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9461): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9470): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9479): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9488): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9497): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9506): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9515): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9524): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9533): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9542): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(54): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(62): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(70): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(78): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(86): error: argument of type "void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(95): error: argument of type "void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(104): error: argument of type "void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(112): error: argument of type "void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(120): error: argument of type "void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(129): error: argument of type "void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(138): error: argument of type "void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(146): error: argument of type "void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10223): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10235): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10247): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10259): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10271): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10283): error: argument of type "const void *" is incompatible with parameter of type "const float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10295): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10307): error: argument of type "const void *" is incompatible with parameter of type "const double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10319): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10331): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10343): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10355): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10367): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10379): error: argument of type "const void *" is incompatible with parameter of type "const int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10391): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10403): error: argument of type "const void *" is incompatible with parameter of type "const long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10413): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10424): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10433): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10444): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10453): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10464): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10473): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10484): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10493): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10504): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10513): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10524): error: argument of type "void *" is incompatible with parameter of type "float *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10533): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10544): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10553): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10564): error: argument of type "void *" is incompatible with parameter of type "double *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10573): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10584): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10593): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10604): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10613): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10624): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10633): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10644): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10653): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10664): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10673): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10684): error: argument of type "void *" is incompatible with parameter of type "int *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10693): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10704): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10713): error: argument of type "void *" is incompatible with parameter of type "long long *"

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h(10724): error: argument of type "void *" is incompatible with parameter of type "long long *"

92 errors detected in the compilation of "/tmp/tmpxft_00008f12_00000000-7_scatter_functor_gpu.cu.cpp1.ii".
ERROR: /scratch/chaimb/tensorflow/tensorflow/core/kernels/BUILD:1140:1: output 'tensorflow/core/kernels/_objs/scatter_functor_gpu/tensorflow/core/kernels/scatter_functor_gpu.cu.pic.o' was not created.
ERROR: /scratch/chaimb/tensorflow/tensorflow/core/kernels/BUILD:1140:1: not all outputs were created or valid.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 147.888s, Critical Path: 69.54s

I've tried disabling most of the options (MKL, architecture optimizations, computability) but the crash happens even with full-default (except CUDA and XLA) configuration.